Nakul Sharma

I'm building cool stuff! Previously, I was a Research Engineer at SpreeAI, where I worked on developing new training schemes and architectures for virtual try-on using diffusion and flow-based models in close collaboration with Dr. Aayush Bansal and Dr. Minh Vo. Prior to that, I completed my Bachelor's in Technology in AI and Data Science at IIT Jodhpur in 2024. At IIT Jodhpur, I was fortunate to be a part of Prof. Anand Mishra's Vision, Language and Learning Group (VL2G), where I worked on 2D generative models, representation learning and multi-modal LLMs.

Email  /  CV  /  Google Scholar  /  GitHub

Research

I'm broadly interested in improving representation learning for various downstream tasks, and multi-modal generative modeling.

This preliminary study introduces a lighweight way to utilize strong semantic representations of visual foundation models for long-tail recognition.

We curate the first large-scale dataset for the task; current models perform poorly due to their visual representations and therefore, we propose to train a weakly supervised visual encoder, PatentMME, using which PatentLMM surpasses GPT-4V significantly.

We propose Partial Discrete Diffusion Process for sketch-guided inpainting, wherein only regions of interest are corrupted during the forward process — aligning the forward process and the reverse process for inpainting.

We study the problem of identifying logos in an open-set setting and propose a to encode textual and visual information from logos, along with a better contrastive loss.

Miscellanea

This website template was ethically stolen from Jon Barron.