LangFuzz: Red Teaming for Language Models

LangFuzz is a command line tool designed to perform red teaming on language model applications and add any points of interest to a LangSmith Dataset.

It generates pairs of similar questions and compares the responses to identify potential failure modes in chatbots or other language model-based systems. For those coming from a software engineering background: this similar to a particular type of fuzz testing called metamorphic testing.

https://github.com/hwchase17/langfuzz

2 comments