Yes, the whole subject of fetishes is pretty incoherent. Some so-called "fetishes" have even reached the level of psychiatric disorders. For example, homosexuality was, until relatively recently (1973), an official "disorder" in the United States.
The Merck Manual defines it like this...