Value iteration and adaptive optimal output regulation with assured convergence rate

Abstract

In this paper, we investigate the learning-based adaptive optimal output regulation problem with convergence rate requirement for disturbed linear continuous-time systems. An adaptive optimal control approach is proposed based on reinforcement learning and adaptive dynamic programming to learn the optimal regulator with assured convergence rate. The above-mentioned problem is successfully solved by tackling a static optimization problem to find the optimal solution to the regulator equations, and a dynamic and constrained optimization problem to obtain the optimal feedback control gain. Without requiring on the accurate system dynamics or a stabilizing feedback control gain, a novel online value iteration algorithm is proposed, which can learn both the optimal feedback control gain and the corresponding feedforward control gain using measurable data. Moreover, the output of the closed-loop system is guaranteed to converge faster or equal to a predefined convergence rate set by user. Finally, the numerical analysis on a LCL coupled inverter-based distributed generation system shows that the proposed approach can achieve desired disturbance rejection and tracking performance.