publications

2026

2025

2024

1. Decomposing and Interpreting Image Representations via Text in ViTs Beyond CLIP

   Sriram Balasubramanian, Samyadeep Basu, and Soheil Feizi

   In The Thirty-eighth Annual Conference on Neural Information Processing Systems 2024

   Abs Link Bib

   Recent work has explored how individual components of the CLIP-ViT model contribute to the final representation by leveraging the shared image-text representation space of CLIP. These components, such as attention heads and MLPs, have been shown to capture distinct image features like shape, color or texture. However, understanding the role of these components in arbitrary vision transformers (ViTs) is challenging. To this end, we introduce a general framework which can identify the roles of various components in ViTs beyond CLIP. Specifically, we (a) automate the decomposition of the final representation into contributions from different model components, and (b) linearly map these contributions to CLIP space to interpret them via text. Additionally, we introduce a novel scoring function to rank components by their importance with respect to specific features. Applying our framework to various ViT variants (e.g. DeiT, DINO, DINOv2, Swin, MaxViT), we gain insights into the roles of different components concerning particular image features. These insights facilitate applications such as image retrieval using text descriptions or reference images, visualizing token importance heatmaps, and mitigating spurious correlations. We release our code to reproduce the experiments in the paper.

   @inproceedings{balasubramanian2024decomposing,
     title = {Decomposing and Interpreting Image Representations via Text in ViTs Beyond {CLIP}},
     author = {Balasubramanian, Sriram and Basu, Samyadeep and Feizi, Soheil},
     booktitle = {The Thirty-eighth Annual Conference on Neural Information Processing Systems},
     year = {2024},
     url = {https://openreview.net/forum?id=Vhh7ONtfvV}
   }

2023

1. Towards Improved Input Masking for Convolutional Neural Networks

   Sriram Balasubramanian, and Soheil Feizi

   In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Oct 2023

   Abs Link Bib

   The ability to remove features from the input of machine learning models is very important to understand and interpret model predictions. However, this is non-trivial for vision models since masking out parts of the input image and replacing them with a baseline color like black or grey typically causes large distribution shifts. Masking may even make the model focus on the masking patterns for its prediction rather than the unmasked portions of the image. In recent work, it has been shown that vision transformers are less affected by such issues as one can simply drop the tokens corresponding to the masked image portions. They are thus more easily interpretable using techniques like LIME which rely on input perturbation. Using the same intuition, we devise a masking technique for CNNs called layer masking, which simulates running the CNN on only the unmasked input. We find that our method is (i) much less disruptive to the model’s output and its intermediate activations, and (ii) much better than commonly used masking techniques for input perturbation based interpretability techniques like LIME. Thus, layer masking is able to close the interpretability gap between CNNs and transformers, and even make CNNs more interpretable in many cases.

   @inproceedings{Balasubramanian_2023_ICCV,
     author = {Balasubramanian, Sriram and Feizi, Soheil},
     title = {Towards Improved Input Masking for Convolutional Neural Networks},
     booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
     month = oct,
     year = {2023},
     pages = {1855-1865}
   }

2. Exploring Geometry of Blind Spots in Vision models

   Sriram Balasubramanian*, Gaurang Sriramanan*, Vinu Sankar Sadasivan, and Soheil Feizi

   In Thirty-seventh Conference on Neural Information Processing Systems (Spotlight) Oct 2023

   Abs Link Bib

   Despite the remarkable success of deep neural networks in a myriad of settings, several works have demonstrated their overwhelming sensitivity to near-imperceptible perturbations, known as adversarial attacks. On the other hand, prior works have also observed that deep networks can be under-sensitive, wherein large-magnitude perturbations in input space do not induce appreciable changes to network activations. In this work, we study in detail the phenomenon of under-sensitivity in vision models such as CNNs and Transformers, and present techniques to study the geometry and extent of “equi-confidence” level sets of such networks. We propose a Level Set Traversal algorithm that iteratively explores regions of high confidence with respect to the input space using orthogonal components of the local gradients. Given a source image, we use this algorithm to identify inputs that lie in the same equi-confidence level set as the source image despite being perceptually similar to arbitrary images from other classes. We further observe that the source image is linearly connected by a high-confidence path to these inputs, uncovering a star-like structure for level sets of deep networks. Furthermore, we attempt to identify and estimate the extent of these connected higher-dimensional regions over which the model maintains a high degree of confidence.

   @inproceedings{balasubramanian2023exploring,
     title = {Exploring Geometry of Blind Spots in Vision models},
     author = {Balasubramanian*, Sriram and Sriramanan*, Gaurang and Sadasivan, Vinu Sankar and Feizi, Soheil},
     booktitle = {Thirty-seventh Conference on Neural Information Processing Systems (Spotlight)},
     year = {2023},
     url = {https://openreview.net/forum?id=uJ3qNIsDGF}
   }

3. Simulating Network Paths with Recurrent Buffering Units

   Divyam Anshumaan*,  Sriram Balasubramanian*, Shubham Tiwari, Nagarajan Natarajan, Sundararajan Sellamanickam, and Venkata N. Padmanabhan

   Proceedings of the AAAI Conference on Artificial Intelligence Jun 2023

   Abs Link Bib

   Simulating physical network paths (e.g., Internet) is a cornerstone research problem in the emerging sub-field of AI-for-networking. We seek a model that generates end-to-end packet delay values in response to the time-varying load offered by a sender, which is typically a function of the previously output delays. The problem setting is unique, and renders the state-of-the-art text and time-series generative models inapplicable or ineffective. We formulate an ML problem at the intersection of dynamical systems, sequential decision making, and time-series modeling. We propose a novel grey-box approach to network simulation that embeds the semantics of physical network path in a new RNN-style model called Recurrent Buffering Unit, providing the interpretability of standard network simulator tools, the power of neural models, the efficiency of SGD-based techniques for learning, and yielding promising results on synthetic and real-world network traces.

   @article{Anshumaan_Balasubramanian_Tiwari_Natarajan_Sellamanickam_Padmanabhan_2023,
     title = {Simulating Network Paths with Recurrent Buffering Units},
     volume = {37},
     url = {https://ojs.aaai.org/index.php/AAAI/article/view/25820},
     doi = {10.1609/aaai.v37i6.25820},
     number = {6},
     journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
     author = {Anshumaan*, Divyam and Balasubramanian*, Sriram and Tiwari, Shubham and Natarajan, Nagarajan and Sellamanickam, Sundararajan and Padmanabhan, Venkata N.},
     year = {2023},
     month = jun,
     pages = {6684-6692}
   }

4. Can AI-Generated Text be Reliably Detected?

   Vinu Sankar Sadasivan, Aounon Kumar,  Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi

   Mar 2023

   Abs Link Bib

   The rapid progress of Large Language Models (LLMs) has made them capable of performing astonishingly well on various tasks including document completion and question answering. The unregulated use of these models, however, can potentially lead to malicious consequences such as plagiarism, generating fake news, spamming, etc. Therefore, reliable detection of AI-generated text can be critical to ensure the responsible use of LLMs. Recent works attempt to tackle this problem either using certain model signatures present in the generated text outputs or by applying watermarking techniques that imprint specific patterns onto them. In this paper, both empirically and theoretically, we show that these detectors are not reliable in practical scenarios. Empirically, we show that paraphrasing attacks, where a light paraphraser is applied on top of the generative text model, can break a whole range of detectors, including the ones using the watermarking schemes as well as neural network-based detectors and zero-shot classifiers. We then provide a theoretical impossibility result indicating that for a sufficiently good language model, even the best-possible detector can only perform marginally better than a random classifier. Finally, we show that even LLMs protected by watermarking schemes can be vulnerable against spoofing attacks where adversarial humans can infer hidden watermarking signatures and add them to their generated text to be detected as text generated by the LLMs, potentially causing reputational damages to their developers. We believe these results can open an honest conversation in the community regarding the ethical and reliable use of AI-generated text.

   @misc{sadasivan2023aigenerated,
     title = {Can AI-Generated Text be Reliably Detected?},
     author = {Sadasivan, Vinu Sankar and Kumar, Aounon and Balasubramanian, Sriram and Wang, Wenxiao and Feizi, Soheil},
     month = mar,
     year = {2023},
     eprint = {2303.11156},
     archiveprefix = {arXiv},
     primaryclass = {cs.CL},
   }

2022

2020

1. What’s in a Name? Are BERT Named Entity Representations just as Good for any other Name?

   Sriram Balasubramanian*, Naman Jain*, Gaurav Jindal*, Abhijeet Awasthi, and Sunita Sarawagi

   In Proceedings of the 5th Workshop on Representation Learning for NLP Jul 2020

   Abs Link Bib

   We evaluate named entity representations of BERT-based NLP models by investigating their robustness to replacements from the same typed class in the input. We highlight that on several tasks while such perturbations are natural, state of the art trained models are surprisingly brittle. The brittleness continues even with the recent entity-aware BERT models. We also try to discern the cause of this non-robustness, considering factors such as tokenization and frequency of occurrence. Then we provide a simple method that ensembles predictions from multiple replacements while jointly modeling the uncertainty of type annotations and label predictions. Experiments on three NLP tasks shows that our method enhances robustness and increases accuracy on both natural and adversarial datasets.

   @inproceedings{balasubramanian-etal-2020-whats,
     equal_authors = {3},
     title = {What{'}s in a Name? Are {BERT} Named Entity Representations just as Good for any other Name?},
     author = {Balasubramanian*, Sriram and Jain*, Naman and Jindal*, Gaurav and Awasthi, Abhijeet and Sarawagi, Sunita},
     booktitle = {Proceedings of the 5th Workshop on Representation Learning for NLP},
     month = jul,
     year = {2020},
     address = {Online},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2020.repl4nlp-1.24},
     doi = {10.18653/v1/2020.repl4nlp-1.24},
     pages = {205--214}
   }