Let's start gently. If someone says "What the heck are you doing?" we know what he or she means. Adding the words "the heck" to the question "what are you doing?" makes clear that the speaker is expressing at least as much criticism as inquiry.
But why does the phrase mean this? How does "the heck" come to mean something like "madness" or "folly" - so that "what the heck are you doing" means "what folly are you committing?"
I'm not sure what "the heck" is. Is there even such a thing as a heck? Probably not, but there is a thing, or at least an idea, of "hell" and pretty clearly "what the heck" is a softened version of "what the hell."
But what does "what the hell" mean? Just reading these three words, by themselves, it's hard to figure out what they could possibly mean. But I guess they are a contraction of a phrase that would have some meaning: "what in the hell." Then the speaker is saying that what you're doing is something that fits in hell.
Fine. But why "what in the hell?" instead of "what in hell"? No one says "go to the hell" or "not till the hell freezes over." And they do sometimes say "what in hell" - those very words, which seem to do the job precisely. So where did the "the" come from? And where did the "in" go? Somehow, it seems, "in" got traded for "the."
There's another problem. We can understand "what the heck/hell" as a somewhat garbled reference to a locale where bad things are to be found. But what about "what the f---?" (I won't spell it out, this being a family-friendly blog.) "The f---" is not a place. I guess the usual human ambivalence about sex could make being "in" or associated with that activity a bad thing. But I still don't see where the "the" comes in.
Of course, we could explain "what the f---" as just a more surly or sexual version of "what the heck/hell," and that phrase, again, as meaning "what in heck/hell." But I wonder whether the whole location explanation (based on the invisible "in") is just after-the-fact rationalization. Maybe people just like to throw nasty words into their sentences, and like it so much that they happily throw grammar and literal meaning out the window in the process.
After all, language serves us - not the other way around. The rules of grammar are rules we make. But I still don't get what the heck the rule is that explains "what the heck."