Method Detail: VLSBench
Back to Leaderboard| Benchmark: | VLSOT |
|---|---|
| Short name: | VLSBench |
| Long name: | Visual Leakage in Multimodal Safety Benchmark |
| Description: | VLSBench is a multimodal safety benchmark designed to evaluate whether Vision-Language Models (VLMs/MLLMs) genuinely use visual information when responding to safety-sensitive image-text inputs. It addresses Visual Safety Information Leakage (VSIL), where the textual prompt unintentionally reveals the harmful content in the image, allowing a model to make a safety decision without actually understanding the image. The benchmark is designed to provide a more reliable evaluation of multimodal safety and cross-modal reasoning. |
| Reference: | Hu, Xuhao, Liu, Dongrui, Li, Hao, Huang, Xuanjing, Shao, Jing, VLSBench: Unveiling Visual Leakage in Multimodal Safety. In arXiv preprint arXiv:2411.19939, 2024. |
| Last submitted: | September 15, 2026 |
| Published: | September 15, 2026 at 07:17:37 |
| Submissions: | 2 |
| Project page / code: | https://github.com/AI45Lab/VLSBench.git |
| Open source: | Yes |
Benchmark performance
| Submission Date | SR@0.5 (↑) | SR@0.7 (↑) | AOR (↑) | PR@1.0 (↑) | ACE (↓) | PR@0.5 (↑) |
|---|---|---|---|---|---|---|
| 2026-09-15 07:29 | - | - | - | - | - | - |
| 2026-09-15 07:25 | - | - | - | - | - | - |