JioHotstar Benchmarks Knowledge Bases for AI Coding Agents
JioHotstar compared six ways for coding agents to find their way around large codebases. Its evaluation found structural code search useful for well-defined engineering tasks, while semantic retrieval helped more with ambiguous requirements.
Adding retrieval layers did not consistently improve task completion or reduce token usage. The team used subsets of SWE-bench and CodeScaleBench alongside a custom question-answering harness for its own codebase, so the results should be read in that context.