Abhisek Keshari
Researcher
Senior Data Scientist at Target
I study what has to hold before a reported result in deep reinforcement learning means what it appears to, starting with how generalization is measured in procedurally generated environments.
About
I graduated from IIT Jammu with a B.Tech in Mechanical Engineering and a CS minor (Director's Gold Medal, 9.1/10 CGPA). My first research experience, on multi-scale feature extraction for image quality assessment, introduced me to the question of how architectural design choices affect model robustness, a thread I now pursue in the reinforcement-learning generalization setting.
After working as a data scientist, formulating real-world allocation and bidding problems as sequential decision-making under constraints, I deepened my RL foundations through independent study (Sutton & Barto, the NPTEL reinforcement learning lectures and Stanford's CS234) and through the ISI Kolkata Winter Schools on Deep Learning (2023 and 2025) and the CNI Summer School on MDPs and POMDPs at IISc (2026). Since January 2026 I have been pursuing independent research on evaluation methodology for RL generalization in procedurally generated environments, targeting Fall 2027 PhD admissions.
Research
I work on what must hold before a reported comparison in deep reinforcement learning means what it appears to.
That means a measured reference point for every return, a stated rule for which policy is evaluated, behavioural checks on convergence, and the unexamined defaults that propagate through a field's shared code. A second thread asks what encoder architectures and auxiliary objectives actually contribute to an agent's representation, and under what training conditions that contribution is detectable at all. I am interested in carrying these questions to RL with large language models, where evaluation choices such as decoding rules, baselines and seed counts are often just as unstated.
Three threads
Evaluation protocol design
How choices papers rarely state (the test-time action rule, the reference point for a return, defaults inherited from shared code) change what a generalization gap measures, and how to build matched-protocol evaluation the field can adopt.
Convergence diagnostics
How much a generalization gap says about transfer depends on whether the policy has learned, checked behaviourally against a random floor, with entropy as a cheap screen rather than a verdict.
Representation and auxiliary objectives
What encoder architectures and auxiliary losses actually contribute to an agent's representation, and whether that contribution is still there by the end of training.
Previously, I worked on multi-scale image quality assessment. MSFPT extends the IQT transformer to run in parallel over features at four image scales; it was our team's entry to the NTIRE 2022 Perceptual IQA Challenge (7th of 8 final-phase teams), described in the CVPR 2022 workshop challenge report.
News
Manuscript on auxiliary-loss gradient liveness in PPO posted: “Is Your Auxiliary Loss Still On?” PDF ↗
Revised version (v2) of the ProcGen generalization-gap preprint on arXiv. arXiv ↗
Preprint on ProcGen evaluation methodology released on arXiv. arXiv ↗
Attended the CNI Summer School on model approximation in MDPs and POMDPs, IISc Bangalore.
Began independent research on evaluation methodology in deep RL.
Promoted to Senior Data Scientist at Target, working on large-scale fulfilment-network optimization.
Selected for the Winter School on Deep Learning at ISI Kolkata.
Began research on transformer-based intrusion detection as a Research Engineer at ISRDC, IIT Bombay.
Co-authored the NTIRE 2022 Challenge report on Perceptual Image Quality Assessment (CVPR Workshop). Paper ↗
Preprint “Multi-Scale Features and Parallel Transformers Based Image Quality Assessment” released. arXiv ↗
Projects
What Does a ProcGen Generalization Gap Measure? Action Rules, Residual Entropy, and the Missing Random Floor
Argues that a generalization gap should be read against a measured random floor. On identical checkpoints, the test-time action rule decides which policy is measured: in miner, the sampled policy scores 5.1× the floor while its argmax scores below it in every run. Also shows that 32–66% of policy entropy sits on functionally equivalent actions, and audits the evaluation code of twelve prior ProcGen codebases.
Is Your Auxiliary Loss Still On? Measuring Gradient Liveness in PPO
Logs how much gradient each auxiliary loss (CURL, SPR, reconstruction) sends to the shared encoder during PPO training, with a hook that leaves training bit-for-bit unchanged. SPR's gradient fades in every run, to a median 0.039 of its early value, tracking early saturation of its moving-average target; a pre-specified prediction of how this scales with the target's momentum failed and is reported as failed. Proposes a 500K-step probe that flags the problem early.
Multi-Scale Features and Parallel Transformers Based Image Quality Assessment
B.Tech research at IIT Jammu, supervised by Prof. Vinit Jakhetiya. Extended the IQT transformer to run identical encoder–decoders in parallel over frozen Inception-ResNet-v2 features at four image scales and average their scores. Built the pipeline and ran 45+ experiments; SRCC 0.977 on LIVE, 0.920 on TID2013 and 0.883 on KADID-10k. Team Pico Zen's entry to the NTIRE 2022 Full-Reference IQA challenge, 7th of 8 final-phase teams.
Publications
Preprints, manuscripts and a co-authored challenge report. Author name in bold.
- Preprint · arXiv
What Does a ProcGen Generalization Gap Measure? Action Rules, Residual Entropy, and the Missing Random Floor
Abhisek Keshari
A generalization gap in reinforcement learning, return on training levels minus return on held-out levels, is usually reported without a reference point. We argue that it should be read against a measured random floor: t…
- Manuscript
Is Your Auxiliary Loss Still On? Measuring Gradient Liveness in PPO
Abhisek Keshari
Auxiliary self-supervised losses in reinforcement learning are judged by the final return of the agent that carries them, which cannot tell an objective that is still supplying gradient from one whose gradient has faded.…
- Preprint · arXiv
Multi-Scale Features and Parallel Transformers Based Image Quality Assessment
Abhisek Keshari*, Komal*, Sadbhawna, Badri Subudhi*Equal contribution.
This paper focuses on image quality assessment in multimedia content, using transformers and a multi-scale feature-extraction approach to account for distortions at different scales. A Siamese network is established betw…
- CVPR 2022 Workshop (NTIRE)
NTIRE 2022 Challenge on Perceptual Image Quality Assessment
J. Gu, H. Cai, C. Dong, J. S. Ren, R. Timofte, …, A. Keshari, …, et al.
This paper reports on the NTIRE 2022 challenge on perceptual image quality assessment (IQA), held in conjunction with the New Trends in Image Restoration and Enhancement workshop (NTIRE) at CVPR 2022. The challenge addre…
Experience
Independent Research
Evaluation methodology in deep RL, on ProcGen. Two sole-author papers: a measured random floor and test-time action rules for generalization gaps, with an audit of twelve public ProcGen codebases; and a gradient-liveness check for auxiliary losses in PPO. Built the full PPO training and evaluation pipeline in PyTorch, CleanRL and Weights & Biases, run on a single consumer GPU.
CNI Summer School
IISc Bangalore
Model approximation in MDPs and POMDPs. Taught by Prof. Aditya Mahajan (McGill / Mila).
Senior Data Scientist
Target Corporation
Formulating large-scale supply-chain optimization (item-level demand allocation across fulfilment centres) as linear programs with operational and capacity constraints.
Winter School on Deep Learning
ISI Kolkata
Nine-week course from neural-network fundamentals through CNNs, RNNs, transformers, graph neural networks and generative models (VAEs, GANs, diffusion) to LLMs, RLHF and reinforcement learning.
Data Scientist
Target Corporation
Formulated large-scale display-advertising budget allocation as a reinforcement learning problem: sequential decision-making with policy optimization under budget and delivery constraints across 11 product categories. Integrated deep-learning response prediction with constrained optimization (LP/MILP solvers), and built a parallel training and data-preparation pipeline for RL experimentation, achieving a 4× reduction in training time. Developed a SKU-level bid recommendation system using distributional modelling and percentile-based optimization.
Research Engineer
ISRDC, IIT Bombay · Prof. Virendra Singh
Surveyed transformer architectures for intrusion detection and formulated a cross-domain problem applying attention to graph-structured network traffic data. Experimented with transformer components on packet-flow and network-log data, framing intrusion detection as sequence modelling, and turned the findings into proposed research problems for the lab. Implemented LeViT-based malware classification on MaleVis, reducing misclassification and inference time over a CNN baseline.
Winter School on Deep Learning
ISI Kolkata
Coursework in deep-learning foundations, optimization, and neural network architectures, the first of two ISI Kolkata winter programs.
Data Scientist
MeraPashu360 (dairy agritech startup)
Formulated workload distribution as a Mixed-Integer Programming problem using Google OR-Tools. Built a warehouse-location optimization framework integrating demand estimation and geospatial modelling (hexagonal tessellation) with MIP-based optimization.
B.Tech, IIT Jammu
Mechanical Engineering · CS Minor
Mechanical Engineering (Major), Computer Science (Minor). CGPA 9.1/10.
Director's Gold Medal for the best all-round performance among all graduating students.
Institute Silver Medal for the highest CGPA in the Mechanical Engineering department.
Talks & Presentations
Transformers in CSec: A Survey
ISRDC, IIT Bombay · 2023
Survey of transformer architectures applied to intrusion and malware detection, presented to the ISRDC lab.
Multi-Scale Features and Parallel Transformers Based Image Quality Assessment
ISRDC, IIT Bombay · 2023
Presented the MSFPT method and its NTIRE 2022 results to the ISRDC lab.
Awards & Achievements
Director's Gold Medal
Indian Institute of Technology, Jammu · Oct 2022
Director's Gold Medal is given to the student for having the best all-round performance among all the graduating students.
Institute Silver Medal
Indian Institute of Technology, Jammu · Oct 2022
Institute Silver Medal is given to the student for having the best academic performance (the highest CGPA) among the graduating students in the department.
Silver Medal, Inter IIT Tech Meet 2021
Indian Institute of Technology, Guwahati · Sep 2021
Awarded Silver Medal at Inter IIT Tech Meet 2021 for outstanding performance in technical competitions.