Research

🚀 Research Vision

I aim to advance the frontier of safe, interpretable, and adaptive AI for cyber-physical systems operating under uncertainty and dynamic constraints. My research sits at the intersection of machine learning, optimization, and control theory, with a particular focus on:

• Physics-informed Deep Reinforcement Learning (DRL)
• Probabilistic & Bayesian Modeling
• Large Language Models (LLMs) for autonomous reasoning
• Vision-based simulation environments

By tightly integrating domain knowledge into learning frameworks, I design agents capable of robust decision-making in real-world, high-stakes environments such as smart grids, robotics, and intelligent infrastructure.

---

🧠 Research Themes

Safe & Trustworthy Reinforcement Learning

🎯 Objective

Develop control agents that guarantee system safety, stability, and robust learning in dynamic, uncertain, and partially observable environments.

🔍 Core Focus Areas

• Constrained policy optimization and reward shaping
• Physics-based priors in DRL
• Adversarial resilience and anomaly detection
• Epistemic and aleatoric uncertainty quantification

Transfer Learning & Meta-Adaptation

🎯 Objective

Enabling rapid generalization across distribution shifts in topology, weather, or load profiles.

🔍 Core Focus Areas

• Transferable actor-critic architectures
• Simulation-to-real (Sim2Real) adaptation
• Meta-RL for sample efficiency

Vision-Simulation Integration

🎯 Objective

Bridge the gap between perception and control by combining synthetic sensors, simulated environments, and end-to-end learning pipelines.

🔍 Core Focus Areas

• Perception-action loops with CARLA, AirSim
• Multi-modal representation fusion (image + state)
• Autonomous control with embedded perception modules
• End-to-end autonomous control pipelines

LLM-Augmented Decision Systems

🎯 Objective

Develop control agents that guarantee system safety, stability, and robust learning in dynamic, uncertain, and partially observable environments.

🔍 Core Focus Areas

• LLMs for summarizing environment states and guiding agents
• Translating textual inputs into actionable policies
• Facilitating human-AI collaboration in dynamic tasks

🔬 Application Domains

Domain	Description
⚡ Smart Energy Systems	Volt-VAR control, DER coordination, and federated DRL for power grid stability
🚘 Autonomous Systems	Safe navigation, adaptive planning, and control in simulation and real-world driving environments
🛡 Secure AI for Infrastructure	Resilience against cyber-attacks and adversarial scenarios in safety-critical systems

---

📚 Selected Publications

📝 Journal Papers

1. Kundan Kumar, Gelli Ravikumar
   Physics-based Deep Reinforcement Learning for Grid-Resilient Volt-VAR Control (Under Review)
   IEEE Transactions on Smart Grid, 2025
   Paper Code Poster

---

🎤 Conference Papers

1. Kundan Kumar, Gelli Ravikumar
   Transfer Learning Enhanced Deep Reinforcement Learning for Volt-Var Control in Smart Grids
   IEEE PES Grid Edge Technologies Conference & Exposition, 2025
   Paper Code Poster

2. Kundan Kumar, Aditya Akilesh Mantha, Gelli Ravikumar
   Bayesian Optimization for Deep Reinforcement Learning in Robust Volt-Var Control
   IEEE PES General Meeting, 2024
   Paper Code Poster

3. Kundan Kumar, Gelli Ravikumar
   Deep RL-based Volt-VAR Control and Attack Resiliency for DER-integrated Distribution Grids
   IEEE ISGT, 2024
   Paper Code Poster

4. JK Francis, C Kumar, J Herrera-Gerena, Kundan Kumar, MJ Darr
   Deep Learning and Pattern-based Methodology for Multivariable Sensor Data Regression
   IEEE ICMLA, 2022
   Paper Code Poster

5. Kin Gwn Lore, Nicholas Sweet, Kundan Kumar, N Ahmed, S Sarkar
   Deep Value of Information Estimators for Collaborative Human-Machine Information Gathering
   ACM/IEEE ICCPS, 2016
   Paper Code Poster

---

🧪 Ongoing Projects

• Federated DRL for Cyber-Resilient Volt-VAR Optimization
  Exploring decentralized, communication-efficient coordination among DERs using LSTM-enhanced PPO agents.

• One-Shot Policy Transfer with Physics Priors
  Training grid agents in simpler environments and adapting to complex grids (IEEE 123-bus, 8500-node) in few iterations.

• LLM-Guided Autonomous Planning for Smart Buildings
  Integrating OpenAI and Claude with CityLearn to convert user prompts into interpretable policy instructions.