Research

Research Vision & Mission

I aim to develop safe, interpretable, and adaptive AI systems for real-world cyber-physical environments that operate under uncertainty, constraints, and adversarial conditions. My research bridges the domains of machine learning, optimization, and control theory, with a strong emphasis on safety, robustness, and generalization.

My work centers around the following pillars:

  • Safe & Trustworthy Reinforcement Learning: Designing agents that are robust to adversarial attacks, resilient to distributional shifts, and capable of safe exploration.
  • Physics-informed Deep Reinforcement Learning (DRL): Embedding physical laws and constraints into learning frameworks for stability, interpretability, and faster convergence.
  • Probabilistic & Bayesian Modeling: Probabilistic & Bayesian Modeling: Capturing both epistemic and aleatoric uncertainties for reliable control in high-stakes, partially observable systems.
  • Large Language Models (LLMs) for autonomous reasoning: Leveraging large language models (LLMs) to enhance planning, explainability, and human-AI collaboration in control systems.
  • Vision-based simulation environments: Using platforms like CARLA and CityLearn to train agents in multimodal, visually rich, and interactive worlds.

By tightly integrating domain knowledge into learning frameworks, I aim to enable resilient, generalizable, and safe AI for critical applications including smart grids, autonomous systems, and intelligent infrastructure.

My Research Focus Areas

DRL-based Control

DRL Control DRL for Volt-VAR
Design control agents for voltage regulation and reactive power optimization in smart distribution grids.

Actor Critic Physics-Informed Actor-Critic
Embed grid physics and control limits directly into the DRL learning loop for stable and efficient decisions.

Sim2Real Transfer Sim-to-Real Transfer
Train agents in simulated OpenDSS environments and deploy them on real-time OPAL-RT setups.

Safe & Trustworthy RL

Safe RL Robust & Stable Learning
Develop agents that ensure system safety, robustness, and interpretability under uncertainty.

Uncertainty Quantification Uncertainty-Aware Policies
Quantify epistemic and aleatoric uncertainty in high-stakes, partially observable settings.

Physics-Informed DRL Physics-Informed DRL
Incorporate physical constraints into DRL agents to ensure safe and interpretable control.

Transfer & Meta-Adaptation

Transfer Learning Domain Adaptation
Enable agents to generalize across grids with different topologies, dynamics, and loads.

Meta RL Meta-RL for Efficiency
Leverage meta-reasoning to accelerate learning in low-data, high-variance scenarios.

Vision-Simulation Integration

Perception Control Perception-Control Fusion
Use CARLA and AirSim to train end-to-end systems in visual RL tasks with sensors.

Multimodal Fusion Multi-modal Representations
Combine visual, state, and contextual features for better decision-making.

LLM-Augmented Decision Systems

LLM for summarization LLM-Guided Control
Translate natural language into actionable policies for real-world environments.

Application Domains

Domain Description
⚡ Smart Energy Systems Volt-VAR control, DER coordination, and federated DRL for power grid stability
🚘 Autonomous Systems Safe navigation, adaptive planning, and control in simulation and real-world environments
🛡 Secure AI for Infrastructure Resilience against cyber-attacks and adversarial scenarios in safety-critical systems

Publications

  1. Kundan Kumar, Gelli Ravikumar
    Physics-based Deep Reinforcement Learning for Grid-Resilient Volt-VAR Control (Under Review)
    IEEE Transactions on Smart Grid, 2025
    Paper Code Poster

  1. Kundan Kumar, Gelli Ravikumar
    Advanced Semi-Supervised Learning With Uncertainty Estimation for Phase Identification in Distribution Systems
    IEEE PES General Meeting, 2025
    Paper Code Poster

  2. Kundan Kumar, Gelli Ravikumar
    Transfer Learning Enhanced Deep Reinforcement Learning for Volt-Var Control in Smart Grids
    IEEE PES Grid Edge Technologies Conference & Exposition, 2025
    Paper Code Poster

  3. Kundan Kumar, Aditya Akilesh Mantha, Gelli Ravikumar
    Bayesian Optimization for Deep Reinforcement Learning in Robust Volt-Var Control
    IEEE PES General Meeting, 2024
    Paper Code Poster

Ongoing Projects

  • Federated DRL for Cyber-Resilient Volt-VAR Optimization
    Decentralized, communication-efficient control using LSTM-enhanced PPO agents across distributed DERs.

  • One-Shot Policy Transfer with Physics Priors
    Train agents on small topologies and adapt to IEEE 123-bus, 8500-node networks in a few iterations.

  • LLM-Guided Autonomous Planning for Smart Buildings
    Convert user prompts to interpretable control policies using LLMs (OpenAI, Claude) in CityLearn environments.