Reinforcement Learning for Dynamic Campaign Budget Optimization

Abstract

In the age of digital marketing, advertisers must adjust their spending across millions of real-time bids and simultaneously react to highly random users. We present a reinforcement learning approach to maximize campaign budgets with independent decision-making by comparing discrete and continuous control. Using the Mendeley Online Advertisement Click-Through Rate data, we simulate the budget allocation process as a Markov Decision Process where each state encodes context, user, and financial information. Two agents Q-Learning and Deep Deterministic Policy Gradient were implemented and evaluated in identical conditions. We show that DDPG is faster, more stable, better click-through rate, and ROI, compared to Q-Learning. Budget-penalty helps keep budgets in check and save resources without harming user interactions. Our findings indicate that reinforcement learning provides an scalable and interpretable foundation for real-time, adaptive marketing budget optimization.