The first time I remember ever understanding what it meant to be lonely was during my first strip abroad.

Yes, "ever" is an adverb. It modifies the verb participle "understanding".


PaulM