I’ve had something similar happen. People immediately jump to the worst possible opinion of you, not realizing they’re damning themselves in the process by making themselves worthy of our scorn. What would you have done if he had said “fire me”?