Multimodal Large Language Models (MLLMs) are susceptible to a newly identified attack vector involving query-relevant images. These images, crafted using advanced techniques such as Stable Diffusion and typography, can circumvent established safety protocols and provoke unsafe outputs, even when the underlying language model is configured for safety. The attack leverages vulnerabilities within the vision-language alignment module, which can be manipulated by image prompts closely associated with malicious textual queries. Mitigation steps: **For AI Developers:** * Implement robust safety prompts that instruct the MLLM to reject malicious queries, ensuring clear communication on the refusal of unsafe or harmful inputs. * Develop and integrate advanced harm detection mechanisms capable of analyzing both textual and visual inputs prior to generating responses. **For Model Trainers/Fine-tuners:** * Enhance the robustness of vision-language alignment modules during training to reduce susceptibility to manipulation by query-relevant images. * Conduct comprehensive adversarial testing using diverse and innovative techniques to identify and mitigate potential vulnerabilities.