I'm not an expert, but UV seems to be a key factor. I think we are about the only species that doesn't see some form of UV light. Since UV shows up as a bright object to birds & animals, even cammo clothes don't help unless treated. Most laundry detergents have brighteners in them. Most dyes used in clothes have UV in them. If you happen to remember the blacklight craze, then you know what we look like to birds & animals.
I use a detergent that has no brighteners and I wash whatever I use for birding several times, to break down the brighteners in the clothes. I have a black light for Mothing, so I use that to check my clothes. That way I don't wear anything that looks bright to birds & animals. To me, that makes a big difference. I wear a lot of denim and I even wear some red & blue sometimes. I know hummingbirds seem to favor colors like red or yellow, but everything I wear is washed several times and then checked by a black light. There are products that you can buy to kill UV in your clothes, but as I say, using a detergent that is free of brighteners works the easiest and cheapest.
Actually, I think keeping a low profile, keeping hidden, no talking and no sudden movements helps more than anything. Compared to a bird, we are huge, so even the best clothing won't do much if we move about in the field.
Also each bird or animal has a different threshold for how much invasion they will tolerate. Some birds show little concern for us and others seem to never come out of the brush. Mating and Nesting seasons have a factor there too. Some species will defend their turf to anything including humans and you can get closer at those times.
There are things in the natural world that show UV light bands. From what I understand some plants actually will show UV to attract birds. I have never gone out with a UV light to find out, it's just what I have read.
These are just my thoughts from what I have experienced.........