Kundan Kumar

AI Safety Research Fellow Β· PhD in Computer Science Β· Iowa State University

I’m an AI researcher with a Ph.D. in Computer Science (minor: Statistics) from Iowa State University, and a current AI Safety Research Fellow at Algoverse.

My path into research started with a practical question: how do we build systems that behave reliably when the real world doesn’t cooperate? That question pulled me from software engineering at Comcast into a PhD focused on safety-critical control for smart energy infrastructure β€” and eventually toward the broader challenge of making AI systems trustworthy at scale.

I work at the intersection of deep reinforcement learning, AI safety, and physical systems β€” building agents that not only perform well, but can be trusted in high-stakes environments.


Current Focus

AI Safety Research Fellow @ Algoverse (Jan 2026 – present)
Designing and evaluating methods for agentic AI safety β€” including behavioral evaluations for reward hacking and deceptive alignment, adversarial robustness frameworks for LLM-based agents, and scalable oversight mechanisms. This builds directly on my doctoral work on physics-informed, safety-critical reinforcement learning.

Research

My doctoral work centered on physics-informed deep reinforcement learning for smart grid control: embedding domain knowledge, safety constraints, and uncertainty directly into the learning process so agents make reliable decisions under distribution shifts and partial observability.

Over time, β€œhow do we keep agents safe?” scaled beyond energy systems. I now focus on AI safety and alignment more broadly:

πŸ”¬
Agentic Evaluations
Designing behavioral and mechanistic tests to detect reward hacking, deceptive alignment, and obfuscated internal states in LLM-based agents.
πŸ›‘οΈ
Adversarial Robustness
Understanding and hardening AI systems against distributional attacks, prompt injection, and specification gaming in both classical and foundation model pipelines.
πŸ”­
Scalable Oversight
Exploring how humans and increasingly capable AI systems can collaborate to maintain meaningful supervision at scale.
⚑
Safe RL for CPS
Physics-informed DRL with Bayesian uncertainty quantification, constraint satisfaction, and transfer learning for smart energy infrastructure.
πŸ€–
LLM-Integrated Agents
Frameworks connecting perception, planning, and language reasoning for autonomous systems β€” bridging interpretable decision-making with low-level control.
πŸ‘οΈ
Autonomous Perception
Vision-based perception (detection, segmentation, sensor fusion) integrated with learning-based control and trajectory planning.

For a full account of projects and publications, see the Research and Projects pages.


Technical Skills

AI / Machine Learning
Deep Reinforcement Learning Physics-Informed Neural Networks Bayesian Deep Learning Uncertainty Quantification Transfer Learning Multi-Agent Systems
AI Safety & Alignment
Adversarial Robustness Agentic Evaluations Scalable Oversight Reward Hacking Detection Deceptive Alignment
Large Language Models & Agents
LangChain / LangGraph RAG Pipelines Tool-Use & Memory Prompt Engineering OpenAI / Anthropic APIs HuggingFace Transformers
Languages & Frameworks
Python PyTorch TensorFlow R OpenAI Gym / Stable Baselines3 CARLA Simulator OpenCV scikit-learn SQL Git / Docker

Education & Experience

Ph.D., Computer Science Iowa State University 2020 – 2026
Minor in Statistics. Dissertation: "Safe and Robust Deep Reinforcement Learning for Constrained Control in DER-Integrated Smart Energy Grids." Advised by Dr. Christopher Quinn and Dr. Ravikumar Gelli. Coursework spanned deep learning, NLP, computer vision, AI for cybersecurity, statistical theory, empirical methods, and experimental design.
Deep RL Safety-Critical Control Uncertainty Quantification Bayesian ML Smart Grids
AI Safety Research Fellow Algoverse Jan 2026 – Present
Developing evaluation frameworks and robustness methods for agentic AI safety, including behavioral tests for reward hacking, deceptive alignment, and scalable human-AI oversight.
AI Safety Agentic Evals Adversarial Robustness
Teaching Assistant Iowa State University 2020 – 2026
Six years supporting 300+ students across software development, database systems, and CS fundamentals. Held office hours, designed assignments, and mentored undergraduate research projects.
Software Engineering Database Systems CS Fundamentals
Software Engineer Comcast Prior to PhD
Worked in industry on software engineering before pivoting to research β€” motivated by the question of how to build systems that behave reliably when the real world doesn't cooperate.
Software Engineering Systems

At a Glance

6+ Years Research
300+ Students Taught
10+ Publications
3 Research Areas

Beyond Research

I try to give back what I learn. I write on AI safety, reinforcement learning, and LLMs on Substack, and produce video explainers and walkthroughs on YouTube. Outside of work, I love cooking and ice skating πŸ›Ό.

View full CV β†’   See my Research β†’