In the sentence: "To be objective in my decision is hard.", Is the infinitive phrase considered noun, an adjective or an adverb?

None of those. It's simply a 'non-finite to-infinitival clause' functioning as the subject of the sentence.

PaulM

Paul is right. However, I would consider it acting as a noun.

Here is a lesson on the roles infinitives play in sentences.

Replace the infinitive phrase with cake. If it works, it's a noun.
Replace the infinitive phrase with big. If it works, it's an adjective.
Replace the infinitive phrase with quickly. If it works, it's an adverb.