1. Bow out.
2. Tell the drummer in a kind way that you prefer to work as a one man band. It is not your job to improve others. Of coure, offer him the whole truth if he asks, but only if he asks.
3. Tell management you're available. Merely say that you prefer to be a one man band and leave it at that.
4. Tell the drummer that if the management calls you that you will take the job but that you aren't going to aggressively pursue work there. It is not necessary to tell the drummer that you left your # with management.
5. If the management hires you, explain that you are obliged to take the work as this is how you earn a living.
If you follow this plan, you have done nothing unethical, you have done nothing harsh. If his feelings are hurt, it is not your fault.
Beakybird