Privacy leakage in multi-agent dialogue
A two-agent dialogue benchmark that isolates what makes LLM agents leak private information, and how the leakage compounds turn by turn.
paper →
02
Projects
Benchmarks, agents, evaluations and the unglamorous infrastructure that makes the experiments possible.
Mila · EQUAL Lab
IDP-Bench
Testing whether LLMs protect people who are implicated by someone else’s data, through the lens of contextual integrity.
paper →IDEAFix
An evaluation framework for creative defixation prompting, which uncovered hive-mind behaviour across models.
paper →Byzantine-robust cooperative MARL
Extending BARDec-POMDP to agents that turn adversarial with probability β — what cooperation still means when a teammate might not be one.
write-up →Leakage evals at scale
A bulk vLLM inference pipeline plus scripts for scoring privacy leakage across model families.
repo →Google DeepMind · AQUA team, 2023–25
Dimension-wise raters for creative outputs
Trained raters for individual quality dimensions using synthetic data generation. The methodology fed into the ICLR 2025 MoFA paper on causal rubrics.
paper →Critique-then-refine creative agents
LLM agents that critique and revise their own creative writing, plus a pairwise evaluation protocol. Reached an 81.25% win rate over a Gemini baseline on a creativity benchmark.
Style transfer with planning agents
Led an unsupervised text style transfer project using LLM-based planning agents, improving on the state of the art by 15–40% across four datasets.
IIT Kharagpur · CNeRG Lab, 2019–23
Financial document understanding
Datasets and models for numeric labelling (FNXL), relation extraction (FinRED), causality extraction and table understanding in financial documents.
FinRED code →Knowledge-graph completion
Knowledge-graph completion with hyperbolic embeddings.
Medical NLI with knowledge graphs
Injecting UMLS knowledge into natural language inference over clinical text (EMNLP 2019).
paper →Microsoft Research India, 2018
EzPC
Built an OCaml-to-C++ transpiler, formalized the EzPC language, and benchmarked it on a decision-tree workload — a secure multi-party computation framework for ML.