Koopman in the Field

Overview

We propose a novel data-driven framework for feedback control of hard-to-model, possibly time-varying, mechanical systems, such as soft robots, that operate in real-world environments and in close interaction with humans. The research in this project is guided by two major research questions:

  1. Control design: Develop novel theoretical foundations in data-driven controller design for nonlinear, time-varying systems to imitate trajectories under input and output safety constrains.
  2. Embodiment design: Develop methods to design the embodiment (hardware and sensors) of the robot to support control and long-term autonomy.
Overview illustration with an idealized soft robot of the three parts of our project: control synthesis, Learning from Demonstrations, Adaptation
Overview illustration with an idealized soft robot of the three parts of our project: control synthesis, Learning from Demonstrations, Adaptation

Motivation

Imagine a worker, in the field on a farm, tasked with deploying a soft robot for the first time. The operator needs an intuitive way to instruct the robot on a given task, without technical knowledge of the machine’s capabilities or limitations. A possible approach is teaching from demonstrations. However, since these robots are flexible and compliant, the user’s demonstrations may contain configurations that the robot cannot reach or maintain on its own; furthermore, real-world field settings can vary dramatically over time in task specifications (different physical characteristics), environmental conditions (temperature and humidity), and repeatability of the robot’s motions (material fatigue).
Our project provides a novel data-driven approach for control and hardware design that models the system and feedback controller jointly to maintain stability, follow desired trajectories, avoid undesired robot configurations, and adapt to changing conditions.

Background on global linearization using the Koopman operator

Consider a general discrete-time autonomous dynamical system \(x^+ = F(x)\). Assume that an explicit model of the system is not available, but only pairs of data \(\{x_i^+,x_i\}\). If the function \(F\) is linear, there exist classical techniques (mostly based on solving linear squares problems) for modeling the behavior of the system. If the function is nonlinear, a possible approach is based on the Koopman operator: in this approach the state \(x\) is lifted to a space of nonlinear observables \(\psi_x(x)\); it can be shown that if the lifted space is large enough (and possibly infinite), we obtain a lifted linear model \(\psi_x^+=K\psi_x\), where \(K\) can be estimated from the data pairs \(\{\psi_x(x_i^+),\psi_x(x_i)\}\).

If the original system is non-autonomous and is input-affine, i.e., \(x^+ = F(x)+G(x)u\), where \(u\) is an input, an application of the same theory leads to a bilinear model \(\psi_x^+=K_{xx}\psi_x+K_{xu}(\psi_x)u\). If the input \(u\) is fixed and does not depend on the state, then we can identify \(K_{xx},K_{xu}\) as in the autonomous case; however, the lifted system does not change the approach for designing a controller where \(u\) is a function of the state \(\psi_x\), since the system is not jointly linear in both.

The Koopman Control Factorization (KCF)

As discussed above, we cannot rely on a naïve pipeline that first models the system, and then designs the controller. Instead, we propose to use a linear-in-the-parameters controller of the form \(u=K_u\psi_u(x)+k_{u}\), where \(\psi_u\) is a set of features extracted from the state \(x\) (or from an output of the system, for an output-feedback controller). We then posit that we should apply Koopman theory to the closed-loop system (which is autonomous). With some technical assumptions on the observables \(\psi_x\) being “rich enough” to “encompass” \(\psi_u\), we then obtain the Koopman Control Factorization (KCF) model \(\psi_x^+=\tilde{K}\psi_x\), where \(\tilde{K}\) depends bilinearly on the traditional model \(K_{xx},K_{xu}\), and the controller parameters \(K_u,k_u\).
The traditional model \(K_{xx},K_{xu}\) can be identified by using random control inputs (a.k.a., motor babbling).
The controller parameters \(K_u,k_u\), instead, depend on the control problem. For stabilizing the system to a desired equilibrium \(\psi_{\textrm{eq}}\), we can write a Lyapunov function \(V(\psi-\psi_{\textrm{eq}})\), and the Lyapunov condition for convergence \(V^+(\psi-\psi_{\textrm{eq}})\leq \lambda V(\psi-\psi_{\textrm{eq}})\). With the KCF, these expressions are quadratic in the control parameters \(K_u,k_u\). We can then use convex optimization to find controller parameters that satisfy the convergence condition.

Animation showing the stabilization of a double inverted pendulum
Results of our KCF design on an inverted double pendulum

Testing on the single and double pendulum

We tested our approach on two simulated systems: a single inverted pendulum, and a double inverted pendulum. Using a fourth-order Runge–Kutta (RK4) method, we simulated 400,000 and 900,000 data pairs for the two sytems respectively.
We then used Semi-Definite Programming (SDP) to design the controllers for both the single and double pendulums. In both cases, our design technique was able to find a stabilizing controller, and we showed that the controller can even generalize to initial conditions not covered by the training set (such as higher initial speeds).

Funding and support

This project is supported by the National Science Foundation grant “Koopman in the Field: Feedback, Teaching, and Adaptation for Real World Mechanical Systems” (Award number 2432394)

Start date: December 23, 2024
End date: January 1, 2025

Disclaimer: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.