This vulnerability pertains to multimodal large language models (MLLMs), where attackers can efficiently perform jailbreaking attacks by utilizing visual inputs to circumvent established safety protocols. The attack methodology involves integrating a visual module into the target LLM, followed by employing a modified Projected Gradient Descent (PGD) algorithm to optimize visual inputs for generating jailbreaking embeddings. These embeddings are subsequently transformed back into textual form and appended to malicious queries, thereby successfully extracting objectionable content from the target LLM. Mitigation steps: **For AI Developers:** * Implement advanced prompt filtering mechanisms to effectively identify and block malicious prompts, including those with visual elements. * Sanitize visual inputs prior to processing by the LLM to ensure the removal of potentially harmful content. * Conduct continuous monitoring of LLMs for indications of jailbreaking attempts and promptly apply updates to mitigate newly identified vulnerabilities. **For Model Trainers/Fine-tuners:** * Incorporate multimodal adversarial training using examples that combine both textual and visual inputs to enhance the model's resilience against attacks.