The purpose of a reputation system is to hold the users of a distributed application accountable for their behavior. The reputation of a user is computed as an aggregate of the feedback provided by fellow users in the system. Truthful feedback is clearly a prerequisite for computing a reputation score that accurately represents the behavior of a user. However, it has been observed that users can hesitate in providing truthful feedback, for example, due to the fear of retaliation. Privacy preserving reputation systems enable users to provide feedback in a private and thus uninhibited manner. In this survey, we propose analysis frameworks for privacy preserving reputation systems. We use these analysis frameworks to review and compare the existing systems in the literature. An emphasis is placed on blockchain-based systems as they are a recent significant development in the area. Utilizing blockchain as a building block, privacy preserving reputation systems have been able to provide properties such as trustlessness, transparency, and immutability, which were absent from prior systems. The results of the analysis reveal several insights and directions for future research. These include exploiting blockchain to its full potential to develop truly trustless systems and to implement some important security properties and defenses against common attacks that are so far ignored by a majority of the systems.