Interesting repo on mm encoder optimization with vllm plugin https://github.com/ddickmann/vllm-factory