They had the relatively recent events of the English Civil Wars and the Glorious Revolution to remind them of the occasional need to remove a head of state (pun intended).

But in those cases, the legislature (Parliament) was the instigator and antagonist of the fight against a tyrant. I guess they never imagined the legislature (Congress) becoming the faithful/cowed lapdog of a tyrant that would consistently fail to use the power the constitution gives it.