Introducing Top Secret

Automatically filter sensitive information before sending it to external services or APIs, such as chatbots and LLMsWe’ve written about how to prevent logging sensitive information when making network requests, but that approach only works if you’re dealing with parameters. What happens when you’re dealing with free text? Filtering the entire string may not be an option if an external API needs to process the value. Think chatbots or LLMs. You could use a regex to filter sensitive information (such as credit card numbers or emails), but that won’t capture everything, since not all sensitive information can be captured with a regex. Fortunately, named-entity recognition (NER) can be used to identify and classify real-world objects, such as a person, or location. Tools like MITIE Ruby make interfacing with NER models trivial.