Post
511
Yesterday, I shared a blog post on generating data for fine-tuning ColPali using the
Qwen/Qwen2-VL-7B-Instruct model.
To simplify testing this approach, I created a Space that lets you generate queries from an input document page image: davanstrien/ColPali-Query-Generator
I think there is much room for improvement, but I'm excited about the potential for relatively small VLMs to create synthetic data.
You can read the original blog post that goes into more detail here: https://danielvanstrien.xyz/posts/post-with-code/colpali/2024-09-23-generate_colpali_dataset.html
To simplify testing this approach, I created a Space that lets you generate queries from an input document page image: davanstrien/ColPali-Query-Generator
I think there is much room for improvement, but I'm excited about the potential for relatively small VLMs to create synthetic data.
You can read the original blog post that goes into more detail here: https://danielvanstrien.xyz/posts/post-with-code/colpali/2024-09-23-generate_colpali_dataset.html