The most effective way to identify red flags is knowing your standards. This is the most fundamental principle in dating and relationships.
A red flag in this relationship isn't necessarily a red flag in that relationship. For example, sleeping with someone else is obviously a red flag in a monogamous relationship. However, it's not a red flag in an open relationship. That's why you have to know your standards.
When something doesn't feel right, ask yourself, "Does this violate my non-negotiable standard?"