Last Updated: 2025-11-12
Total Papers: 50
2025
CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment
Authors: Xue Jiang, Yihong Dong, Mengyang Liu et al. Published: 2025-10-21 arXiv: 2510.18471v1 PDF: Download
Abstract: While Large Language Models (LLMs) excel at code generation by learning from vast code corpora, a fundamental semantic gap remains between their training on textual patterns and the goal of functional correctness, which is governed by formal execution semantics. Reinforcement Learning with Verifiabl…
TritonRL: Training LLMs to Think and Code Triton Without Cheating
Authors: Jiin Woo, Shaowei Zhu, Allen Nie et al. Published: 2025-10-18 arXiv: 2510.17891v1 PDF: Download
Abstract: With the rapid evolution of large language models (LLMs), the demand for automated, high-performance system kernels has emerged as a key enabler for accelerating development and deployment. We introduce TritonRL, a domain-specialized LLM for Triton kernel generation, trained with a novel training fr…
Language Models For Generalised PDDL Planning: Synthesising Sound and Programmatic Policies
Authors: Dillon Z. Chen, Johannes Zenn, Tristan Cinquin et al. Published: 2025-08-25 arXiv: 2508.18507v1 PDF: Download
Abstract: We study the usage of language models (LMs) for planning over world models specified in the Planning Domain Definition Language (PDDL). We prompt LMs to generate Python programs that serve as generalised policies for solving PDDL problems from a given domain. Notably, our approach synthesises polici…
Agent-RLVR: Training Software Engineering Agents via Guidance and Environment Rewards
Authors: Jeff Da, Clinton Wang, Xiang Deng et al. Published: 2025-06-13 arXiv: 2506.11425v2 PDF: Download
Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) has been widely adopted as the de facto method for enhancing the reasoning capabilities of large language models and has demonstrated notable success in verifiable domains like math and competitive programming tasks. However, the efficacy of RLVR…
CodeContests+: High-Quality Test Case Generation for Competitive Programming
Authors: Zihan Wang, Siyao Liu, Yang Sun et al. Published: 2025-06-06 arXiv: 2506.05817v1 PDF: Download
Abstract: Competitive programming, due to its high reasoning difficulty and precise correctness feedback, has become a key task for both training and evaluating the reasoning capabilities of large language models (LLMs). However, while a large amount of public problem data, such as problem statements and solu…
QiMeng-CodeV-R1: Reasoning-Enhanced Verilog Generation
Authors: Yaoyu Zhu, Di Huang, Hanqi Lyu et al. Published: 2025-05-30 arXiv: 2505.24183v4 PDF: Download
Abstract: Large language models (LLMs) trained via reinforcement learning with verifiable reward (RLVR) have achieved breakthroughs on tasks with explicit, automatable verification, such as software programming and mathematical problems. Extending RLVR to electronic design automation (EDA), especially automat…
SATURN: SAT-based Reinforcement Learning to Unleash Language Model Reasoning
Authors: Huanyu Liu, Jia Li, Hao Zhu et al. Published: 2025-05-22 arXiv: 2505.16368v2 PDF: Download
Abstract: How to design reinforcement learning (RL) tasks that effectively unleash the reasoning capability of large language models (LLMs) remains an open question. Existing RL tasks (e.g., math, programming, and constructing reasoning tasks) suffer from three key limitations: (1) Scalability. They rely heav…
Code-Driven Planning in Grid Worlds with Large Language Models
Authors: Ashwath Vaithinathan Aravindan, Zhisheng Tang, Mayank Kejriwal Published: 2025-05-15 arXiv: 2505.10749v1 PDF: Download
Abstract: We propose an iterative programmatic planning (IPP) framework for solving grid-based tasks by synthesizing interpretable agent policies expressed in code using large language models (LLMs). Instead of relying on traditional search or reinforcement learning, our approach uses code generation as polic…
Integrating Symbolic Execution into the Fine-Tuning of Code-Generating LLMs
Authors: Marina Sakharova, Abhinav Anand, Mira Mezini Published: 2025-04-21 arXiv: 2504.15210v2 PDF: Download
Abstract: Code-generating Large Language Models (LLMs) have become essential tools in modern software development, enhancing productivity and accelerating development. This paper aims to investigate the fine-tuning of code-generating LLMs using Reinforcement Learning and Direct Preference Optimization, furthe…
Executable Functional Abstractions: Inferring Generative Programs for Advanced Math Problems
Authors: Zaid Khan, Elias Stengel-Eskin, Archiki Prasad et al. Published: 2025-04-14 arXiv: 2504.09763v2 PDF: Download
Abstract: Scientists often infer abstract procedures from specific instances of problems and use the abstractions to generate new, related instances. For example, programs encoding the formal rules and properties of a system have been useful in fields ranging from reinforcement learning (procedural environmen…
Synthesizing world models for bilevel planning
Authors: Zergham Ahmed, Joshua B. Tenenbaum, Christopher J. Bates et al. Published: 2025-03-26 arXiv: 2503.20124v2 PDF: Download
Abstract: Modern reinforcement learning (RL) systems have demonstrated remarkable capabilities in complex environments, such as video games. However, they still fall short of achieving human-like sample efficiency and adaptability when learning new domains. Theory-based reinforcement learning (TBRL) is an alg…
A Graph-Enhanced Deep-Reinforcement Learning Framework for the Aircraft Landing Problem
Authors: Vatsal Maru Published: 2025-02-18 arXiv: 2502.12617v2 PDF: Download
Abstract: The Aircraft Landing Problem (ALP) is one of the challenging problems in aircraft transportation and management. The challenge is to schedule the arriving aircraft in a sequence so that the cost and delays are optimized. There are various solution approaches to solving this problem, most of which ar…
ACECODER: Acing Coder RL via Automated Test-Case Synthesis
Authors: Huaye Zeng, Dongfu Jiang, Haozhe Wang et al. Published: 2025-02-03 arXiv: 2502.01718v4 PDF: Download
Abstract: Most progress in recent coder models has been driven by supervised fine-tuning (SFT), while the potential of reinforcement learning (RL) remains largely unexplored, primarily due to the lack of reliable reward data/model in the code domain. In this paper, we address this challenge by leveraging auto…
Competitive Programming with Large Reasoning Models
Authors: OpenAI, :, Ahmed El-Kishky et al. Published: 2025-02-03 arXiv: 2502.06807v2 PDF: Download
Abstract: We show that reinforcement learning applied to large language models (LLMs) significantly boosts performance on complex coding and reasoning tasks. Additionally, we compare two general-purpose reasoning models - OpenAI o1 and an early checkpoint of o3 - with a domain-specific system, o1-ioi, which u…
2024
ViSymRe: Vision-guided Multimodal Symbolic Regression
Authors: Da Li, Junping Yin, Jin Xu et al. Published: 2024-12-15 arXiv: 2412.11139v2 PDF: Download
Abstract: Extracting simple mathematical expression from an observational dataset to describe complex natural phenomena is one of the core objectives of artificial intelligence (AI). This field is known as symbolic regression (SR). Traditional SR models are based on genetic programming (GP) or reinforcement l…
Optimizing AI-Assisted Code Generation
Authors: Simon Torka, Sahin Albayrak Published: 2024-12-14 arXiv: 2412.10953v1 PDF: Download
Abstract: In recent years, the rise of AI-assisted code-generation tools has significantly transformed software development. While code generators have mainly been used to support conventional software development, their use will be extended to powerful and secure AI systems. Systems capable of generating cod…
Programming with AI: Evaluating ChatGPT, Gemini, AlphaCode, and GitHub Copilot for Programmers
Authors: Md Kamrul Siam, Huanying Gu, Jerry Q. Cheng Published: 2024-11-14 arXiv: 2411.09224v1 PDF: Download
Abstract: Our everyday lives now heavily rely on artificial intelligence (AI) powered large language models (LLMs). Like regular users, programmers are also benefiting from the newest large language models. In response to the critical role that AI models play in modern software development, this study present…
Reinforcement learning with learned gadgets to tackle hard quantum problems on real hardware
Authors: Akash Kundu, Leopoldo Sarra Published: 2024-10-31 arXiv: 2411.00230v2 PDF: Download
Abstract: Designing quantum circuits for specific tasks is challenging due to the exponential growth of the state space. We introduce gadget reinforcement learning (GRL), which integrates reinforcement learning with program synthesis to automatically generate and incorporate composite gates (gadgets) into the…
Exploring RL-based LLM Training for Formal Language Tasks with Programmed Rewards
Authors: Alexander G. Padula, Dennis J. N. J. Soemers Published: 2024-10-22 arXiv: 2410.17126v1 PDF: Download
Abstract: Proximal Policy Optimization (PPO) is commonly used in Reinforcement Learning from Human Feedback to align large language models (LLMs) with downstream tasks. This paper investigates the feasibility of using PPO for direct reinforcement learning (RL) from explicitly programmed reward signals, as opp…
Language-Model-Assisted Bi-Level Programming for Reward Learning from Internet Videos
Authors: Harsh Mahesheka, Zhixian Xie, Zhaoran Wang et al. Published: 2024-10-11 arXiv: 2410.09286v1 PDF: Download
Abstract: Learning from Demonstrations, particularly from biological experts like humans and animals, often encounters significant data acquisition challenges. While recent approaches leverage internet videos for learning, they require complex, task-specific pipelines to extract and retarget motion data for t…
RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning
Authors: Jonas Gehring, Kunhao Zheng, Jade Copet et al. Published: 2024-10-02 arXiv: 2410.02089v2 PDF: Download
Abstract: Large language models (LLMs) deployed as agents solve user-specified tasks over multiple steps while keeping the required manual engagement to a minimum. Crucially, such LLMs need to ground their generations in any feedback obtained to reliably achieve the desired outcomes. We propose an end-to-end …
In Search of Trees: Decision-Tree Policy Synthesis for Black-Box Systems via Search
Authors: Emir Demirović, Christian Schilling, Anna Lukina Published: 2024-09-05 arXiv: 2409.03260v2 PDF: Download
Abstract: Decision trees, owing to their interpretability, are attractive as control policies for (dynamical) systems. Unfortunately, constructing, or synthesising, such policies is a challenging task. Previous approaches do so by imitating a neural-network policy, approximating a tabular policy obtained via …
AlphaDou: High-Performance End-to-End Doudizhu AI Integrating Bidding
Authors: Chang Lei, Huan Lei Published: 2024-07-14 arXiv: 2407.10279v2 PDF: Download
Abstract: Artificial intelligence for card games has long been a popular topic in AI research. In recent years, complex card games like Mahjong and Texas Hold’em have been solved, with corresponding AI programs reaching the level of human experts. However, the game of Doudizhu presents significant challenges …
RLSF: Fine-tuning LLMs via Symbolic Feedback
Authors: Piyush Jha, Prithwish Jana, Pranavkrishna Suresh et al. Published: 2024-05-26 arXiv: 2405.16661v3 PDF: Download
Abstract: Large Language Models (LLMs) have transformed AI but often struggle with tasks that require domain-specific reasoning and logical alignment. Traditional fine-tuning methods do not leverage the vast amount of symbolic domain-knowledge available to us via symbolic reasoning tools (e.g., provers), and …
Learning to Stop Cut Generation for Efficient Mixed-Integer Linear Programming
Authors: Haotian Ling, Zhihai Wang, Jie Wang Published: 2024-01-31 arXiv: 2401.17527v2 PDF: Download
Abstract: Cutting planes (cuts) play an important role in solving mixed-integer linear programs (MILPs), as they significantly tighten the dual bounds and improve the solving performance. A key problem for cuts is when to stop cuts generation, which is important for the efficiency of solving MILPs. However, m…
LLMs for Relational Reasoning: How Far are We?
Authors: Zhiming Li, Yushi Cao, Xiufeng Xu et al. Published: 2024-01-17 arXiv: 2401.09042v1 PDF: Download
Abstract: Large language models (LLMs) have revolutionized many areas (e.g. natural language processing, software engineering, etc.) by achieving state-of-the-art performance on extensive downstream tasks. Aiming to achieve robust and general artificial intelligence, there has been a surge of interest in inve…
2023
DanZero+: Dominating the GuanDan Game through Reinforcement Learning
Authors: Youpeng Zhao, Yudong Lu, Jian Zhao et al. Published: 2023-12-05 arXiv: 2312.02561v1 PDF: Download
Abstract: The utilization of artificial intelligence (AI) in card games has been a well-explored subject within AI research for an extensive period. Recent advancements have propelled AI programs to showcase expertise in intricate card games such as Mahjong, DouDizhu, and Texas Hold’em. In this work, we aim t…
Program Machine Policy: Addressing Long-Horizon Tasks by Integrating Program Synthesis and State Machines
Authors: Yu-An Lin, Chen-Tao Lee, Guan-Ting Liu et al. Published: 2023-11-27 arXiv: 2311.15960v2 PDF: Download
Abstract: Deep reinforcement learning (deep RL) excels in various domains but lacks generalizability and interpretability. On the other hand, programmatic RL methods (Trivedi et al., 2021; Liu et al., 2023) reformulate RL tasks as synthesizing interpretable programs that can be executed in the environments. D…
LLM-Assisted Code Cleaning For Training Accurate Code Generators
Authors: Naman Jain, Tianjun Zhang, Wei-Lin Chiang et al. Published: 2023-11-25 arXiv: 2311.14904v1 PDF: Download
Abstract: Natural language to code generation is an important application area of LLMs and has received wide attention from the community. The majority of relevant studies have exclusively concentrated on increasing the quantity and functional correctness of training sets while disregarding other stylistic el…
Logic-Q: Improving Deep Reinforcement Learning-based Quantitative Trading via Program Sketch-based Tuning
Authors: Zhiming Li, Junzhe Jiang, Yushi Cao et al. Published: 2023-10-09 arXiv: 2310.05551v3 PDF: Download
Abstract: Deep reinforcement learning (DRL) has revolutionized quantitative trading (Q-trading) by achieving decent performance without significant human expert knowledge. Despite its achievements, we observe that the current state-of-the-art DRL models are still ineffective in identifying the market trends, …
Learning of Generalizable and Interpretable Knowledge in Grid-Based Reinforcement Learning Environments
Authors: Manuel Eberhardinger, Johannes Maucher, Setareh Maghsudi Published: 2023-09-07 arXiv: 2309.03651v1 PDF: Download
Abstract: Understanding the interactions of agents trained with deep reinforcement learning is crucial for deploying agents in games or the real world. In the former, unreasonable actions confuse players. In the latter, that effect is even more significant, as unexpected behavior cause accidents with potentia…
Code Detection for Hardware Acceleration Using Large Language Models
Authors: Pablo Antonio Martínez, Gregorio Bernabé, José Manuel García Published: 2023-07-19 arXiv: 2307.10348v1 PDF: Download
Abstract: Large language models (LLMs) have been massively applied to many tasks, often surpassing state-of-the-art approaches. While their effectiveness in code generation has been extensively studied (e.g., AlphaCode), their potential for code detection remains unexplored. This work presents the first ana…
Continuous-Time Reinforcement Learning: New Design Algorithms with Theoretical Insights and Performance Guarantees
Authors: Brent A. Wallace, Jennie Si Published: 2023-07-18 arXiv: 2307.08920v1 PDF: Download
Abstract: Continuous-time nonlinear optimal control problems hold great promise in real-world applications. After decades of development, reinforcement learning (RL) has achieved some of the greatest successes as a general nonlinear control design method. However, a recent comprehensive analysis of state-of-t…
Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review
Authors: Man Fai Wong, Shangxin Guo, Ching Nam Hang et al. Published: 2023-07-04 arXiv: 2307.02503v1 PDF: Download
Abstract: This paper provides a comprehensive review of the literature concerning the utilization of Natural Language Processing (NLP) techniques, with a particular focus on transformer-based large language models (LLMs) trained using Big Code, within the domain of AI-assisted programming tasks. LLMs, augment…
Neural Task Synthesis for Visual Programming
Authors: Victor-Alexandru Pădurean, Georgios Tzannetos, Adish Singla Published: 2023-05-26 arXiv: 2305.18342v3 PDF: Download
Abstract: Generative neural models hold great promise in enhancing programming education by synthesizing new content. We seek to design neural models that can automatically generate programming tasks for a given specification in the context of visual programming domains. Despite the recent successes of large …
ChatGPT for PLC/DCS Control Logic Generation
Authors: Heiko Koziolek, Sten Gruener, Virendra Ashiwal Published: 2023-05-25 arXiv: 2305.15809v1 PDF: Download
Abstract: Large language models (LLMs) providing generative AI have become popular to support software engineers in creating, summarizing, optimizing, and documenting source code. It is still unknown how LLMs can support control engineers using typical control programming languages in programming tasks. Resea…
From Copilot to Pilot: Towards AI Supported Software Development
Authors: Rohith Pudari, Neil A. Ernst Published: 2023-03-07 arXiv: 2303.04142v1 PDF: Download
Abstract: AI-supported programming has arrived, as shown by the introduction and successes of large language models for code, such as Copilot/Codex (Github/OpenAI) and AlphaCode (DeepMind). Above human average performance on programming challenges is now possible. However, software engineering is much more th…
2022
Symbolic Visual Reinforcement Learning: A Scalable Framework with Object-Level Abstraction and Differentiable Expression Search
Authors: Wenqing Zheng, S P Sharan, Zhiwen Fan et al. Published: 2022-12-30 arXiv: 2212.14849v1 PDF: Download
Abstract: Learning efficient and interpretable policies has been a challenging task in reinforcement learning (RL), particularly in the visual RL setting with complex scenes. While neural networks have achieved competitive performance, the resulting policies are often over-parameterized black boxes that are d…
Parsel: Algorithmic Reasoning with Language Models by Composing Decompositions
Authors: Eric Zelikman, Qian Huang, Gabriel Poesia et al. Published: 2022-12-20 arXiv: 2212.10561v3 PDF: Download
Abstract: Despite recent success in large language model (LLM) reasoning, LLMs struggle with hierarchical multi-step reasoning tasks like generating complex programs. For these tasks, humans often start with a high-level algorithmic design and implement each part gradually. We introduce Parsel, a framework en…
Pac-Man Pete: An extensible framework for building AI in VEX Robotics
Authors: Jacob Zietek, Nicholas Wade, Cole Roberts et al. Published: 2022-11-25 arXiv: 2211.14385v1 PDF: Download
Abstract: This technical report details VEX Robotics team BLRSAI’s development of a fully autonomous robot for VEX Robotics’ Tipping Point AI Competition. We identify and develop three separate critical components. This includes a Unity simulation and reinforcement learning model training pipeline, a malleabl…
UNSAT Solver Synthesis via Monte Carlo Forest Search
Authors: Chris Cameron, Jason Hartford, Taylor Lundy et al. Published: 2022-11-22 arXiv: 2211.12581v3 PDF: Download
Abstract: We introduce Monte Carlo Forest Search (MCFS), a class of reinforcement learning (RL) algorithms for learning policies in {tree MDPs}, for which policy execution involves traversing an exponential-sized tree. Examples of such problems include proving unsatisfiability of a SAT formula; counting the n…
Harfang3D Dog-Fight Sandbox: A Reinforcement Learning Research Platform for the Customized Control Tasks of Fighter Aircrafts
Authors: Muhammed Murat Özbek, Süleyman Yıldırım, Muhammet Aksoy et al. Published: 2022-10-13 arXiv: 2210.07282v1 PDF: Download
Abstract: The advent of deep learning (DL) gave rise to significant breakthroughs in Reinforcement Learning (RL) research. Deep Reinforcement Learning (DRL) algorithms have reached super-human level skills when applied to vision-based control problems as such in Atari 2600 games where environment states were …
Follow-up Attention: An Empirical Study of Developer and Neural Model Code Exploration
Authors: Matteo Paltenghi, Rahul Pandita, Austin Z. Henley et al. Published: 2022-10-11 arXiv: 2210.05506v2 PDF: Download
Abstract: Recent neural models of code, such as OpenAI Codex and AlphaCode, have demonstrated remarkable proficiency at code generation due to the underlying attention mechanism. However, it often remains unclear how the models actually process code, and to what extent their reasoning and the way their attent…
An Empirical Evaluation of Competitive Programming AI: A Case Study of AlphaCode
Authors: Sila Lertbanjongngam, Bodin Chinthanet, Takashi Ishio et al. Published: 2022-08-18 arXiv: 2208.08603v2 PDF: Download
Abstract: AlphaCode is a code generation system for assisting software developers in solving competitive programming problems using natural language problem descriptions. Despite the advantages of the code generating system, the open source community expressed concerns about practicality and data licensing. H…
What is it like to program with artificial intelligence?
Authors: Advait Sarkar, Andrew D. Gordon, Carina Negreanu et al. Published: 2022-08-12 arXiv: 2208.06213v2 PDF: Download
Abstract: Large language models, such as OpenAI’s codex and Deepmind’s AlphaCode, can generate code to solve a variety of problems expressed in natural language. This technology has already been commercialised in at least one widely-used programming editor extension: GitHub Copilot. In this paper, we explor…
Meta-learning from Learning Curves Challenge: Lessons learned from the First Round and Design of the Second Round
Authors: Manh Hung Nguyen, Lisheng Sun, Nathan Grinsztajn et al. Published: 2022-08-04 arXiv: 2208.02821v1 PDF: Download
Abstract: Meta-learning from learning curves is an important yet often neglected research area in the Machine Learning community. We introduce a series of Reinforcement Learning-based meta-learning challenges, in which an agent searches for the best suited algorithm for a given dataset, based on feedback of l…
Graph-based Reinforcement Learning meets Mixed Integer Programs: An application to 3D robot assembly discovery
Authors: Niklas Funk, Svenja Menzenbach, Georgia Chalvatzaki et al. Published: 2022-03-08 arXiv: 2203.04120v2 PDF: Download
Abstract: Robot assembly discovery is a challenging problem that lives at the intersection of resource allocation and motion planning. The goal is to combine a predefined set of objects to form something new while considering task execution with the robot-in-the-loop. In this work, we tackle the problem of bu…
Competition-Level Code Generation with AlphaCode
Authors: Yujia Li, David Choi, Junyoung Chung et al. Published: 2022-02-08 arXiv: 2203.07814v1 PDF: Download
Abstract: Programming is a powerful and ubiquitous problem-solving tool. Developing systems that can assist programmers or even generate programs independently could make programming more productive and accessible, yet so far incorporating innovations in AI has proven challenging. Recent large-scale language …
An Improved Reinforcement Learning Algorithm for Learning to Branch
Authors: Qingyu Qu, Xijun Li, Yunfan Zhou et al. Published: 2022-01-17 arXiv: 2201.06213v1 PDF: Download
Abstract: Most combinatorial optimization problems can be formulated as mixed integer linear programming (MILP), in which branch-and-bound (B&B) is a general and widely used method. Recently, learning to branch has become a hot research topic in the intersection of machine learning and combinatorial optimiza…
Multi-echelon Supply Chains with Uncertain Seasonal Demands and Lead Times Using Deep Reinforcement Learning
Authors: Julio César Alves, Geraldo Robson Mateus Published: 2022-01-12 arXiv: 2201.04651v1 PDF: Download
Abstract: We address the problem of production planning and distribution in multi-echelon supply chains. We consider uncertain demands and lead times which makes the problem stochastic and non-linear. A Markov Decision Process formulation and a Non-linear Programming model are presented. As a sequential decis…