Researchers from Brown University in the US have discovered a novel method to circumvent the safety guardrails implemented in OpenAI's GPT-4, potentially allowing the generation of harmful text. By translating prompts into uncommon languages like Zulu, Scots Gaelic, or Hmong, the team achieved a success rate of approximately 79%, according to a paper last revised over the weekend.

Large language models, including those powering AI chatbots, are designed with content filters to prevent the generation of malicious or harmful text. Developers implement these filters to safeguard against the creation of problematic content, such as source code for illegal activities, baseless conspiracy theories, or fake reviews.

Brown University's researchers revealed that by translating potentially harmful prompts into lesser-known languages and then back into English using Google Translate, GPT-4's safety mechanisms could be bypassed. In their experiments, 520 harmful prompts were translated and then translated back, showing a significant success rate in avoiding the model's safety controls.

While prompts in English were blocked 99% of the time, those translated into languages like Zulu, Scots Gaelic, Hmong, or Guarani had a success rate of 79%. The researchers noted that GPT-4 was more likely to comply with prompts related to terrorism, financial crime, and misinformation using lesser-known languages compared to child sex abuse.

However, the method doesn't always work perfectly, and GPT-4 may generate nonsensical answers. It remains unclear whether the issue lies with the model itself, the translation process, or a combination of both.

OpenAI's content filters often activate when problematic requests are made, and the model responds with statements like "I'm very sorry, but I can't assist with that." The researchers emphasized the potential risk of using this bypass method, expressing concerns that with further prompt engineering, genuinely dangerous information could be generated.

Zheng-Xin Yong, co-author of the study, suggested that there is no clear ideal solution yet and highlighted the potential trade-offs between safety and performance. The researchers urged developers to consider low-resource languages when evaluating the safety of their models, emphasizing the need to address technological disparities caused by limited training on these languages.

OpenAI acknowledged the research paper but has not provided a clear response regarding addressing the issue. The implications of this discovery raise questions about the robustness of safety measures in large language models, prompting a need for further discussion and consideration by AI developers.