The Doctor’s arrogance is a consistent story-generating problem: think of that moment in ‘The Pandorica Opens’ when Matt Smith’s Eleventh Doctor climbs up onto the altar stone and delivers a bombastic speech to the assembled aliens in their spaceships above, except that he’s about to walk into a trap that’s been laid by the aforementioned aliens and this speech is a colossal act of hubris. The Doctor fails, the universe ends and he has to sacrifice himself to save it. After the crescendo of Twelve’s ‘Heaven Sent’, the Doctor acts so unlike himself he acknowledges that he’s not exactly the Doctor right now (with the script for ‘Hell Bent’ indicating that he’s becoming more like the War Doctor the soldiers on Gallifrey knew) . The redemption comes when he realizes he’s gone too far and is willing to put himself through what he’d intended for others.
With Clara and Bill, the Doctor ends up taking over their loss, so that there’s been some form of sacrifice to restore them from death. It’s a riff on a very old story, that of the journey to the Underworld to retrieve something lost (so established that there is a word for it – katabasis). It’s the sacrifice element that enabled former showrunner Steven Moffat to get away with a temporary loss, usually represented by the death/fridging of the companion, because he ultimately substitutes a story where the companion gets to live and the Doctor falls.
Reverting to Format
This substitution happens partly because Moffat prefers a happy ending, but also because Doctor Who cannot sustain the kind of storytelling where companions die for very long (The Doctor though? They can take it, we can put all this loss onto them and they will always come back from it). This is simply because the format of the show dictates it – if the Doctor’s friends died all the time or if it wasn’t fun for them then why would they travel with the Doctor? This is a story arc in the very first series: the Doctor’s behavior is initially off-putting to potential companions, the first cases of the character’s failings generating stories.
However, if the Doctor fails consistently, the show can stretch credulity (especially if it doesn’t address the resulting change in mood). Crucially Doctor Who quickly realized it couldn’t sustain the abrasive relationship between the Doctor, Ian and Barbara at its very beginning because it would become increasingly implausible that the latter would remain on board: Doctor Who would be a show about people torn between wanting to return home and wanting to escape their kidnapper.
Once established, the format of the show always snapped back into place after sustained periods of the Doctor failing – the trick is to make this reset feel satisfying. When the show fails to revert to format it runs into trouble. The cumulative effect of several stories with high death tolls and grim, dirty climaxes gave the Fifth Doctor stories a loose arc which concluded with his regeneration of him (where he manages to save his companion Peri in the process). Again there’s an element of sacrifice involved that makes this feel like redemption.
It was a strange choice, then, to follow this natural stopping point in the Doctor’s failures by aggressively ramping them up, not least by having him strangle Peri and generally treat her terribly. It makes Peri’s continued traveling with the Doctor seem like she’s trapped. Compare this to the Twelfth Doctor’s tragic ‘duty of care’ reasoning for trying to save Clara.