Suspend your suspension of disbelief for just a moment, and this article by E. Paul Zehr on why Bane and Joker are Batman's toughest opponents is impressive in its logic. Zehr, a movement researcher, was at the receiving end of this Q&A which I read years ago, titled Why Batman Could Exist--But Not for Long.

Zehr explains that Bane and Joker exploit a flaw in Batman's crime fighting strategy. Whereas police officers always respond with one level of force above that used by criminals, Batman responds in kind. Also, Batman does not like to kill but instead exploits involuntary human responses to pain.

To avoid killing, Batman uses his opponents’ bodies against them to evoke protective reactions. Nociceptors are receptors detecting actual or impending tissue damage. They relay this information to the spinal cord where they evoke very powerful defensive responses. If you have ever stepped on a very sharp rock while walking barefoot or accidentally touched a hot stove top you will remember the rapid pullback you had of your foot or hand. These signals also arrive in the brain where they may be interpreted as “pain.”
Batman, in the tradition of martial artists the world over, uses those defensive responses to manipulate his opponents. He hurts rather than harms and tries to intimidate rather than inflict permanent damage. Making a use of force continuum work requires extreme skill, poise, and confidence. It also requires an intact and normally functioning nervous system in your opponent.

Said normally functioning nervous system being something that Joker and Bane lack.