IndiaFinBench: An Evaluation Benchmark for Large Language Model Performance on Indian Financial Regulatory Text

The benchmark's difficulty comes not just from domain specificity but from the structure of Indian regulatory language itself: SEBI and RBI documents layer circular references, amendment histories, and jurisdiction-specific numerical thresholds in ways that generic financial benchmarks don't capture. The 406-question count is modest, but the expert annotation and multi-task design (including contradiction detection across documents) is what makes it harder to game than single-task evals.

This arrives in a busy week for financial and domain-specific LLM evaluation. QuantCode-Bench, covered here on April 16, took a similar approach to scoping a narrow financial domain (algorithmic trading strategy generation) with a comparably sized task set (400 tasks). Both papers reflect the same underlying pressure: general benchmarks don't tell practitioners whether a model is actually deployable in a regulated, domain-specific context. The difference is that IndiaFinBench tests comprehension and reasoning over existing regulatory text rather than code generation, which means its failure modes will look more like hallucinated citations than broken syntax.

Watch whether Indian fintech firms or compliance vendors publicly adopt IndiaFinBench as a procurement filter within the next six months. Adoption by even one named institution would signal the benchmark has operational weight beyond academic citation.