Could this be considered correct? "I shan't have done that!"
In the right context, yes; but NOT as a substitute for "I should not have done that!"
"I still have a lot of years to live, and a lot of living to do. I expect I will look back when I'm sixty at all the bad things I've done in my life; but of all the bad things that people do, I shan't have done that!"
"I should not have done that" is Present Perfect Tense. Start with:
"I have done that."
Pictorially, it is:
……|D<…have done…<|NOW…
D = what you did
NOW = the moment of speaking these words.
The speaker looks back from NOW, over a period of time, to a point in the past where this started (point D).
Similarly:
……|D<…shouldn't have done…<|NOW…
'shan't have done': is Future Perfect:
…NOW|<…shan't have done……<|60
The person sees themselves looking back, at the age of 60, over the period of time between when he's 60, and NOW.
Note: while 'should' is still common, most people disregard 'shall' and use 'will' instead.