In a performence or at a performance?

Both actually is very unusual.
But if you want to put them this way, they mean totally different things.
1. I failed in a performance.
meant You failed to meet your standard or requirement in (WHERE) a performance (like dancing, singing)
2. I failed at a performance.
Failed at = what I was failing for ...
I failed at performance ... = I failed at my (singing)/(dancing) performance.

I think it is rare to say " fail " in or at a performance if the word means " a show ". You only fail in a test, exam or contest.
For " performance ", we usually rate it by adjectives like " good " or " bad ".

eg. My performance last night was poor.
His excellent performance won tremendous applause.

It's my opinion.