Valeo

Items

1. Apr 14

   Self-supervised tasks boost visual instruction tuning

   Valeo and ETH Zurich find multimodal LLMs lean on language priors during instruction tuning, and that mixing in 3-10% visually grounded self-supervised tasks phrased as instructions consistently improves vision-centric benchmarks with no architecture changes or extra annotation.