{"id":213,"date":"2023-04-17T19:54:51","date_gmt":"2023-04-17T23:54:51","guid":{"rendered":"https:\/\/sites.bu.edu\/tron\/?p=213"},"modified":"2023-04-17T20:14:47","modified_gmt":"2023-04-18T00:14:47","slug":"unified-vision-based-motion-estimation-and-control-for-multiple-and-complex-robots","status":"publish","type":"post","link":"https:\/\/sites.bu.edu\/tron\/2023\/04\/17\/unified-vision-based-motion-estimation-and-control-for-multiple-and-complex-robots\/","title":{"rendered":"Unified Vision-Based Motion Estimation and Control for Multiple and Complex Robots"},"content":{"rendered":"<h3>Overview<\/h3>\n<p>The project considers teams of robots that need to collaborate on a physical task. As a driving application, we consider a <em>Robot Construction Crew<\/em> that needs to assemble a building using prefabricated components. Each robot might have non-trivial kinematics (e.g., a robotic arm mounted on a non-holonomic platform), and is generally equipped with vision sensors (cameras). To achieve their mission, the robots need to:<\/p>\n<ol>\n<li>Localize themselves with respect to each other, to the construction area, and to the prefabricated components that need to be assembled.<\/li>\n<li>Coordinate their motions to collaboratively grasp, transport and assemble the components<\/li>\n<li>Plan how to survey and inspect the results of their work.<\/li>\n<\/ol>\n<p><center><img loading=\"lazy\" src=\"\/tron\/files\/2023\/04\/formationDesiredDownwardFoVGrid-e1681776728629.png\" alt=\"\" class=\"alignnone size-full wp-image-217\" width=\"325\" height=\"300\" \/> <img loading=\"lazy\" src=\"\/tron\/files\/2023\/04\/gridQuadrotorFovSequence-e1681776718888.png\" alt=\"\" class=\"alignnone size-full wp-image-218\" width=\"352\" height=\"300\" \/> <img loading=\"lazy\" src=\"\/tron\/files\/2023\/04\/kukaCollaborative-e1681776707462.png\" alt=\"\" class=\"alignnone size-full wp-image-223\" width=\"283\" height=\"300\" \/><\/center>The functions above imply vision-kinodynamic constraints deriving from the intrinsic geometric properties of how the robots move and sense. The main insight of this work is that the constraints above are <em>bilinear<\/em> in the variables parametrizing the kinematic configurations of the robots and the objects, i.e., in the translations and rotation matrices.<\/p>\n<p>We introduce novel optimization formulations that take advantage of this structure by combining the vision-kinodynamic constraints into <em>convex<\/em> constraints (e.g., defining a Semi-Definite Program) and <em>non-convex low-rank<\/em> constraints (from the bilinearity over translations and rotations). We expect that this methodology will surpass current state- of-the-art solutions, by providing robust joint solvers instead of cumbersome and brittle pipelines.<\/p>\n<h3><strong>Funding and support<\/strong><\/h3>\n<p><img loading=\"lazy\" src=\"https:\/\/www.nsf.gov\/images\/logos\/nsf1.gif\" alt=\"NSF Logo\" class=\"size-medium alignright\" width=\"166\" height=\"167\" \/>This project is supported by the National Science Foundation grant <a href=\"https:\/\/www.nsf.gov\/awardsearch\/showAward?AWD_ID=2212051&amp;HistoricalAwards=false\">&#8220;Unified Vision-Based Motion Estimation and Control for Multiple and Complex Robots&#8221; (Award number 2212051)<\/a><\/p>\n<p>Start date: August 1, 2022<br \/>\nEnd date: July 31, 2025<\/p>\n<p><span><em>Disclaimer: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.<\/em><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Overview The project considers teams of robots that need to collaborate on a physical task. As a driving application, we consider a Robot Construction Crew that needs to assemble a building using prefabricated components. Each robot might have non-trivial kinematics (e.g., a robotic arm mounted on a non-holonomic platform), and is generally equipped with vision [&hellip;]<\/p>\n","protected":false},"author":11510,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[4],"tags":[],"_links":{"self":[{"href":"https:\/\/sites.bu.edu\/tron\/wp-json\/wp\/v2\/posts\/213"}],"collection":[{"href":"https:\/\/sites.bu.edu\/tron\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sites.bu.edu\/tron\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sites.bu.edu\/tron\/wp-json\/wp\/v2\/users\/11510"}],"replies":[{"embeddable":true,"href":"https:\/\/sites.bu.edu\/tron\/wp-json\/wp\/v2\/comments?post=213"}],"version-history":[{"count":6,"href":"https:\/\/sites.bu.edu\/tron\/wp-json\/wp\/v2\/posts\/213\/revisions"}],"predecessor-version":[{"id":221,"href":"https:\/\/sites.bu.edu\/tron\/wp-json\/wp\/v2\/posts\/213\/revisions\/221"}],"wp:attachment":[{"href":"https:\/\/sites.bu.edu\/tron\/wp-json\/wp\/v2\/media?parent=213"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sites.bu.edu\/tron\/wp-json\/wp\/v2\/categories?post=213"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sites.bu.edu\/tron\/wp-json\/wp\/v2\/tags?post=213"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}