No, you can't use "were" in your example. We give the name "irrealis" to this use of "were"; it's a special mood form sometimes called (incorrectly) the past subjunctive, but it's inappropriate here.
So, "was" is fine. It's a preterite (past tense) form, but the meaning has to do with modality (non-factuality) rather than past time. The preterite "was" in conjuction with preterite "knew" in your example suggests that you don't know who is going, not that you didn't know who was going. When a verb is used this way, it's called a modal preterite.
Incidentally, irrealis "were" is used in expressions like "If I were you", "I wish she were here", and the like. But in any case, irrealis "were" is an untidy relic of an earlier system, and some speakers usually, if not always, use preterite "was" instead.
Yes, I got it. The only thing that is still bothering me is the following
... you don't know who is going, not that you didn't know who was going ...
The first half makes sense; for the second, do you mean that ..."you didn't know who was going" is the truth about " ... if I knew who else were going"?
The idea is that "I didn't know who was going to it", despite the use of preterite (past tense) verb-forms, refers to the situation at the time of utterance, i.e. the present, cf:
(1) I would have gone to the party but I didn't know who was going to it. (past tense verb-form "was" and past time situation)
(2) I'd go to the party if I knew who else was going to it. (past tense verb-forms but present situation)
In (1) "was" is a straightforward past tense auxiliary verb indicating past time. You may now know who actually attended the party, but you didn't at the time. By contrast, the use of the modal preterites "knew" and "was" in (2) reflect modality, or counterfactuality, at the time of the utterance, not at some time in the past.