Digital preview controller design using reinforcement learning

Abstract

This paper discusses the design of a preview LQ controller. We assume a linear time-invariant continuous model of controlled mechatronics subsystem. The model is discretized using a generalized hold function (GHF), which is defined by its impulse response. This approach can improve the closed loop gain margin, among other benefits. The plant model is then expanded to include the preview part of the system, and the LQ controller design methodology is applied. An incremental control signal is introduced in the criterion, which adds integral action to the controller. The digital controller is obtained offline as a solution to the Riccati equation. This solution provides both the feedback and feedforward controllers. Finally, using adaptive dynamic programming, we convert the offline procedure into an online procedure, resulting in an intelligent controller. The analysis is supported by an illustrative example involving a general mechatronic subsystem. The results highlight the potential of designed digital controller to significantly enhance the performance of manufacturing systems. The obtained controller can be related to the ILC (Iterative Learning Control) family of control algorithms, which are typically used in manufacturing systems.