Abstract

Model predictive control (MPC) [1] is an established optimization-based control technique, which is widely used for systems subject to input and state constraints. When dealing with iterative tasks, past data can be leveraged to enhance the safety and performance of the controller [2] and estimate model uncertainties. This paper investigates an MPC formulation that ensures robust constraint satisfaction while improving performance by adapting to unknown parameters and learning from previous iterations.

Robust MPC schemes [3, 4] ensure robust constraint satisfaction for bounded model mismatch. In particular, tube-based robust MPC formulations [5] provide such robustness guarantees with a favorable computational complexity.

These approaches achieve robust constraint satisfaction by enclosing all possible trajectories within a tube around the nominal trajectory using a local feedback law. While tube-based robust MPC performs well under disturbances, it can be overly conservative in the presence of constant parametric uncertainties. This limitation has increased interest in the development of robust adaptive MPC, where uncertain parameters are adapted online using past data [6].

In [7], safety and performance are decoupled using two separate models, where only the performance model is adapted. Set membership estimation techniques have been used to update models while establishing recursive feasibility and stability for time-invariant, time-variant, and stochastic finite impulse response systems [8, 9, 10]. However, these methods are applicable only to a limited class of asymptotically stable systems. This motivated the authors of [11] to extend these results to a broader class of linear state-space systems. However, the resulting robust adaptive MPC formulation requires optimization over an increasing number of variables and constraints. To address this, the authors in [12] and [13] developed robust adaptive MPC algorithms with fixed computational complexity during runtime. Furthermore, [14] demonstrates an experimental application of [12] to a quadrotor system.

In many applications, the same control task is repeatedly solved, making it beneficial to leverage information from previous iterations in the controller design. Iterative Learning Control [15] addresses this by refining the controller in each iteration.  [16, 17] develop an iterative learning MPC scheme that utilizes past experiments to define a terminal cost and a terminal set. By including these terminal ingredients in the MPC formulation, convergence to an improved closed-loop cost function by learning a safety set and a cost-to-go function is shown. The recent work [18] combines iterative learning and system identification for Lipschitz-continuous nonlinear systems. However, this approach results in a large computational demand. A modular framework for learning-based control of uncertain linear systems is proposed in [19]. The approach ensures robust safety by iteratively refining a safe set using robust tubes and improved model estimates. In [10], set membership estimation and iterative learning are combined for an iterative task with a linear system. However, this method is limited to constant parametric offsets and cannot treat general parametric uncertainty and disturbances.

We propose a new robust adaptive learning MPC (RALMPC) algorithm that combines the robust adaptive MPC framework from[12] with the iterative learning approach of [17]. The main idea is to adaptively estimate the uncertain parameters online using set membership estimation while simultaneously refining the control policy for iterative tasks using data from previous iterations. The algorithm is designed for linear state-space systems subject to constant parametric uncertainties and additive bounded disturbances. Figure 1 shows a schematic overview of the different elements of the algorithm.

The key contributions of this work are as follows:

• A robust adaptive learning MPC algorithm is developed for linear time-invariant systems with parametric uncertainties and additive disturbances performing iterative tasks.
• A robust control invariant set is constructed using previously predicted trajectories. The set is iteratively expanded by adding new trajectories. Additionally, a terminal cost based on the worst-case cost to go of the predicted trajectories is used. This approach enables the controller to (robustly) improve the closed-loop cost over iterations.
• The proposed method ensures recursive feasibility and robust constraint satisfaction. In addition, a robust bound on the closed-loop cost is shown.
• A numerical example is provided to demonstrate the effectiveness of the proposed algorithm, comparing it with the approach in [12]. The results highlight the advantages of a shorter prediction horizon, leading to reduced computational burden, as well as improved closed-loop cost due to the improvement of terminal conditions over iterations.

The remainder of this paper is structured as follows. Section 2 presents the problem formulation. Section 3 introduces the parameter learning, constraint tightening, and terminal conditions. Section 4 provides a theoretical analysis of the resulting MPC scheme. Section 5 presents numerical simulations, followed by concluding remarks in Section 6.

Notation: We denote \( [A]_i\) as the \( i\) -th row of matrix \( A\) . The quadratic norm with the positive definite matrix \( Q \succ 0\) is defined as \( \|x\|_Q^2 = x^T Q x\) . The unit hypercube is denoted by \( \mathbb{B}_p = \{\theta \in \mathbb{R}^p |  \lVert \theta \rVert_{\infty} \leq 1 \}\) , and the Minkowski sum is denoted by \( A \oplus B\) . The cardinality of set \( A\) is denoted by \( |A|\) . \( \mathcal{K}_\infty\) is a class of functions \( \alpha\) : \( \mathbb{R}_{\geq0}\rightarrow\mathbb{R}_{\geq0}\) , which are continuous, strictly increasing, unbounded and satisfy \( \alpha(0)=0\) .

Consider the discrete-time linear uncertain system

where \( x_t \in \mathbb{R}^n\) and \( u_t \in \mathbb{R}^m\) are state and input at time \( t\in\mathbb{N}\) . The system is affected by an additive disturbance \( d_t \in \mathbb{R}^n\) and an unknown constant parameter \( \theta=\theta^* \in \mathbb{R}^p\) .
We consider mixed state and input constraints of the form

where \( \mathcal{Z}= \{(x,u) \in \mathbb{R}^{n+m} \mid F_j x + G_j u \leq 1, \, j = 1, \dots, q\}\) is a compact set. Moreover, the uncertainties and the disturbance satisfy the following assumption.

Throughout this paper, we focus on robustly stabilizing the origin. We consider an iterative scenario and assume we start from the same initial state \( x_s\) at each iteration. Each successful steering of the system from \( x_s\) to the origin is considered a successful iteration, denoted by \( h\in\mathbb{N}\) . Our goal is to design a controller that improves the control performance over iterations while robustly ensuring safety. Conceptually, we consider the following infinite horizon control problem:

where \( \mathbb{X} ^h_{t}\) is a tube that outer bounds all possible state trajectories, \( \Theta^{\text{HC},h} \subseteq \Theta_0^{\text{HC}}\) is the hypercubic parameter set at the \( h\) -iteration, and \( u^h_{t}(x)\) is some feedback policy. We consider a quadratic stage cost \( \ell(x, u) = \|x\|_Q^2 + \|u\|_R^2\) where Q and R are positive definite. Since problem (4) is not tractable, we present a robust adaptive learning MPC scheme that approximates the above problem in the next section.

In Section 3.1, we introduce the framework for parameter adaptation. This is followed by the introduction of the polytopic tube based on the previous work of [12] in Section 3.2 and the worst-case stage cost in Section 3.3. The robust adaptive learning MPC algorithm is presented in Section 3.4. Section 3.5 deals with learning terminal conditions, which is the main theoretical contribution of this paper. Finally, Section 3.6 presents the offline and online computation algorithms.

3.1 Set membership estimation

3.2 Polytopic tube

3.3 Worst-case stage cost

3.4 Robust adaptive learning MPC scheme

3.5 Terminal conditions

3.6 Offline and Online Algorithms

In this section, we show that the proposed MPC scheme is recursively feasible, ensures constraint satisfaction, and achieves a desirable closed-loop performance bound. First, we ensure robust control invariance of the learned terminal set \( \mathcal{CS}^h_{\text{robust}}\) :

The following theorem utilizes the properties of the learned terminal set and cost (Prop. 1) to establish the closed-loop properties of the proposed MPC scheme.

The decrease condition (45) of the optimal cost with the positive definiteness quadratic cost \( \ell\) mirrors the Lyapunov condition to show practical asymptotic stability of the closed-loop system [21, Th. 2.20]. Hence, given also suitable lower and upper bounds on the optimal cost \( J^{\text{LMPC},h,*}\) , Theorem 1 also implies practical asymptotic stability. However, a formal proof, including possible additional required technical conditions, is beyond the scope of this paper.

The following example illustrates the performance and computational efficiency improvements of the proposed robust adaptive learning MPC scheme compared to robust adaptive MPC(RAMPC) [12] for an iterative task.
We consider a mass spring damper system

with the fixed mass \( m=1\) , the uncertain damper constant \( c \in [0.1, 0.3]\) and the spring constant \( k \in [0.5, 1.5]\) . Additionally, the system is affected by an additive disturbance \( |d_t| \leq 0.2\) . The unknown true values are denoted as \( c^* = 0.3\) and \( k^* = 0.5\) . Moreover, the moving window \( M\) is set to \( 10\) for all experiments. We use Euler discretization with a sampling time \( T_s=0.1\) . Transforming the second-order ODE into a state-space model yields the state \( x = (x_1, \dot{x}_1) \in \mathbb{R}^2\) . Then the constraints are \( \mathcal{Z} = [-0.2, 4.1] \times [-5, 5] \times [-15, 15]\) . The control goal is to iteratively steer the system from \( x_s=\begin{bmatrix}4 & 0\end{bmatrix}^T\) to the origin, while minimizing the quadratic stage cost with \( Q=\text{diag}(1,10^{-2})\) and \( R=10^{-1}\) .
The controller \( K\) is computed using linear matrix inequalities [12]. Furthermore, the polytope \( \mathcal{P}\) is the maximal \( \rho\) -contractive set for the constraints. The initial solution of the Assumption 4 for the convex safe set (26) is computed by solving Problem (22) with a finite horizon and robust positively invariant terminal set based on [12, Proposition 3]. In the terminal set, we append steps until we reach the steady state in Assumption 4.

First, we investigate how changing the horizon \( N\) affects the computation time and the resulting cost in comparison to RAMPC. To obtain a comparable result over the iterations, we set the disturbances \( d_t\) to a constant value \( d_t=0.1\) . Both algorithms are repeated over 20 iterations, where the set \( \Theta^{\text{HC}}\) is further reduced from iteration to iteration. We solve the problem using CasADi[22] and IPOPT, and the code is publicly available online https://github.com/HannesPetrenz/RALMPC_Linear_Uncertain_Systems. The RAMPC algorithm requires a horizon of at least \( N=22\) steps to be feasible and we set the horizon to \( N_{\text{RAMPC}}=25\) . Table 1 shows an overall improvement in the cost of the final iteration for \( N_{\text{RALMPC}}=\{8,12,18\}\) and average CPU time compared to RAMPC. The improvement in computation time is due to the reduced horizon, which has a larger impact than the additional optimization variable \( \lambda\) . Note that the dimension of \( \lambda\) grows with the size of the sample set, emphasizing the importance of reducing the sample set to only promising states. Table 1 shows the trade-off between cost improvement and computation time increase in comparison to the optimal solution (OS), which solves the robust infinite horizon optimization problem with knowledge of the true parameter, accounting only robustly for the unknown disturbance \( d_t\)

Figure 2 shows the closed-loop trajectories for the different controllers. The horizon length of the RALMPC is set to \( N_{\text{RALMPC}}=12\) because this is the best trade-off between computation time and cost. It clearly shows that the RLAMPC algorithm approaches the optimal solution through learning. Moreover, it reveals that even if the horizon \( N_{\text{RAMPC}}\) is twice times larger, the trajectory of RAMPC is far from the optimal trajectory, which demonstrates the advantage of the RALMPC.

We proposed a robust adaptive learning MPC framework for linear systems with parametric uncertainties and additive disturbances. We extended the robust adaptive MPC in [12] to iterative tasks by learning the terminal cost and the terminal set. The proposed algorithm is recursively feasible with robust constraint satisfaction and guarantees a robust bound on the closed-loop cost. In a numerical example, we show the advantages of the method for iterative tasks in comparison to [12]. The RALMPC decreases the closed-loop cost function by iterative learning. Moreover, it allows to reduce the horizon of the MPC, which leads to a reduced computational complexity.