Unified Vision-Based Motion Estimation and Control for Multiple and Complex Robots
Overview
The project considers teams of robots that need to collaborate on a physical task. As a driving application, we consider a Robot Construction Crew that needs to assemble a building using prefabricated components. Each robot might have non-trivial kinematics (e.g., a robotic arm mounted on a non-holonomic platform), and is generally equipped with vision sensors (cameras). To achieve their mission, the robots need to:
- Localize themselves with respect to each other, to the construction area, and to the prefabricated components that need to be assembled.
- Coordinate their motions to collaboratively grasp, transport and assemble the components
- Plan how to survey and inspect the results of their work.
We introduce novel optimization formulations that take advantage of this structure by combining the vision-kinodynamic constraints into convex constraints (e.g., defining a Semi-Definite Program) and non-convex low-rank constraints (from the bilinearity over translations and rotations). We expect that this methodology will surpass current state- of-the-art solutions, by providing robust joint solvers instead of cumbersome and brittle pipelines.
Funding and support
This project is supported by the National Science Foundation grant “Unified Vision-Based Motion Estimation and Control for Multiple and Complex Robots” (Award number 2212051)
Start date: August 1, 2022
End date: July 31, 2025
Disclaimer: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.