Why is the word little wrong when it is used as an adjective?

It isn't.

I saw a little dog.

That's fine.

I suspect this is a question relating to indefinite pronouns somehow. Can you elaborate?