The Chain-of-Jailbreak (CoJ) attack is a sophisticated method designed to circumvent safety protocols in image generation models. This attack operates by fragmenting a malicious query into a series of innocuous sub-queries. Each sub-query prompts the model to incrementally edit the image, ultimately leading to the creation of harmful content. The attack exploits various editing operations such as insertion, deletion, and modification of elements, including words, characters, and images. Through this iterative process, the model's safeguards are systematically bypassed, allowing for the generation of content that would typically be restricted. Mitigation steps: **For AI Developers:** * Implement enhanced content filtering mechanisms to detect and mitigate harmful image content, accounting for sequences of edits. * Utilize think-twice prompting strategies to instruct the model to evaluate the potential safety of the generated image prior to completion. **For Model Trainers/Fine-tuners:** * Improve contextual safety by training models to assess the cumulative effect of multiple queries and edits, understanding the sequence of events leading to potential harm. * Integrate adversarial training techniques by expanding datasets with examples of CoJ attacks to enhance model robustness against manipulation.