Since it modifies the subject "I", why isn't the clause an adjectival?
Because it doesn't actually modify the subject "I", but simply gives some information about it; there is a difference. Adjuncts (adverbials) modify verbs, not nouns. But note, though, that participial clauses can modify nouns:
(1) "The person calling me was very rude".
(2) "The person who was calling me was very rude".
In those examples, the underlined clauses are modifying the noun "person". Such clauses function rather like adjectives, but they are best called modifiers.