# Research projects

My research interests lie at the intersection of automatic control, robotics and computer vision. I am particularly interested in applications of Riemannian geometry and in distributed problems involving teams of multiple agents.

## Robust, Scalable, Distributed Semantic Mapping for Search-and-Rescue and Manufacturing Co-Robots

### Overview

The goal of this project is to enable multiple co-robots to map and understand the environment they are in to efficiently collaborate among themselves and with human operators in education, medical assistance, agriculture, and manufacturing applications. The first distinctive characteristic of this project is that the environment will be modeled semantically, that is, it will contain human-interpretable labels (e.g., object category names) in addition to geometric data. This will be achieved through a novel, robust integration of methods from both computer vision and robotics, allowing easier communications between robots and humans in the field. The second distinctive characteristic of this project is that the increased computation load due to the addition of human-interpretable information will be handled by judiciously approximating and spreading the computations across the entire network. The novel developed methods will be evaluated by emulating real-world scenarios in manufacturing and for search-and-rescue operations, leading to potential benefits for large segments of the society. The project will include opportunities for training students at the high-school, undergraduate, and graduate levels by promoting the development of marketable skills.

This is a joint project with Dario Pompili at Rutgets University.

### Publications

**Funding and support**

This project is supported by the National Science Foundation grant “Robust, Scalable, Distributed Semantic Mapping for Search-and-Rescue and Manufacturing Co-Robots” (Award number 1734454).

*Disclaimer: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.*

## Control of Micro Aerial Vehicles under Aerodynamic and Physical Contact Interactions

### Overview

The goal of this project is to make quadrotors and other similar small-scale flying rotorcraft safer and easier to fly. Both recreational and commercial use of these vehicles has recently surged in popularity. However, safety concerns about potentially damaging collisions limit their deployment near people or in close formation, and the current state of the art in vehicle control is insufficient for potential applications involving flight inside of complicated structures such as industrial plants, forests and caves. This project will lead to innovations in control schemes, aerodynamic interactions modeling, and robust aerial vehicles design. Together, these innovations will make small-scale vehicles less likely to cause unintended damage, suitable for use in extreme environments such as caves, and more easily piloted. This will allow in turn the use of these vehicles in new industrial monitoring and search-and-rescue applications, thus bringing the benefits of these platforms to larger segments of society.

### Contraction theory and Riemannian tangent bundles

We started by considering attitude-only controllers; in this case, the state space can be modeled as the tangent bundle of the Special Orthogonal Lie group (i.e., as the space of rotations together with their angular velocities). Building upon previous work, the metric and the differentiation operator that are used in standard contraction theory for Euclidean spaces correspond, respectively to the Riemannian metric and covariant differentiation on Riemannian manifolds, and the differentiation operation corresponds to covariant differentiation. However, existing results for covariant differentiation on tangent bundles (which is what is required for our application) can be applied only to a specific type of metric (the Sasaki metric, which is only a particular case of natural metrics). The first step under this grant, therefore, has been to generalize existing results to a more general class of non-natural metrics. While previous natural metrics, intuitively, do not allow “interaction” between changes in rotations and changes in velocity, our new class of metrics instead explicitly allows such “interactions”. These results are a necessary step for the application of the contraction theory framework for proving and optimizing convergence of controllers on Riemannian manifolds.

### Experimental aerodynamic characterization of quadrotors

One of the objectives of the project is to develop low-order models that better capture the aerodynamic effects in the system due to its geometry, operating conditions, and presence of nearby surfaces. These models will be developed by a combined use of experimental data and detailed simulations. In this regard, we have developed an experimental platform for collecting and characterizing the relation between commands given to the rotors in an aerial vehicle (quadrotor), and the actual forces and torques generated. The setup involves an industrial-grade 6-D force-torque sensor, and a custom quadrotor platform (based on the PixHawk microcontroller).

**Funding and support**

This project is supported by the National Science Foundation grant “Control of Micro Aerial Vehicles under Aerodynamic and Physical Contact Interactions” (Award number 1728277).

*Disclaimer: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.*

## Distributed semantic processing in camera networks

**Overview**

In many applications, sensor networks can be used to monitor large geographical regions. This typically produces large quantities of data that need to be associated, summarized and classified in order to arrive to a semantically meaningful descriptions of the phenomena being monitored. The long-term guiding vision of this project is a distributed network that can perform this analysis autonomously, over long periods of times, and in a scalable way. As a concrete application, this research focuses on smart camera networks with nodes that are either static or part of robotic agents. The planned work will result in systems that are more efficient, accurate, and resilient. The algorithms developed will find wide applications, including in security (continuously detecting suspicious individuals in real time) and the Internet of Things. As part of the broader impacts, the project will produce educational material to explain the scientific results of the project to a K12 audience.

**QuickMatch: Fast Multi-Image Matching via Density-Based Clustering**

The first result of this project is an algorithm, QuickMatch, that performs consistent matching across multiple images. Quickmatch formulates the problem as a clustering problem (see figure) and then uses a modified density-based algorithm to separate the points in clusters that represents consistent matches across images.

In particular, with respect to previous work, QuickMatch 1) represents a novel application of density-based clustering; 2) directly outputs consistent multi-image matches without explicit pre-processing (e.g., initial pairwise decisions) or post-processing (e.g., thresholding of a matrix); 3) is non-iterative, deterministic, and initialization-free; 4) produces better results in a small fraction of the time (it is up to 62 times faster in some benchmarks); 5) can scale to large datasets that previous methods cannot handle (it has been tested with more than 20k+ features); 6) takes advantage of the distinctiveness of the descriptor as done in traditional matching to counteract the problem of repeated structures; 7) does not assume a one-to-one correspondence of features between images; 8) does not need a-priori knowledge of the number of entities (i.e., clusters) present in the images. Code is available under the Software page.

**“NetMatch: the Game”, and educational board game**

We developed an alpha version of a board game, called NetMatch, that provides a tangible and fun way to explain the main research challenges in the project. This game is for two to four players, whose goal is to move their pawns across a network (one hop at a time) from the edges in order to match pawns with similar symbols. When all the pawns for a symbol are matched, a letter for a secret word is revealed. The player that discovers all the letters of his word is the winner.

To start playing, simply download, print, and cut the pieces from the PDF document.

The majority of the game’s components components (board, cards, pawns) are procedurally generated. The code is made freely available, so that it is possible to easily generate variations of the game. The code made available on a git repository (https://bitbucket.org/tronroberto/pythonnetmatchgame).

If you play the game, and have comments or suggestions, please email them to tron@bu.edu.

**Publications and other resources**

You can also follow the journey of one of the undergraduate students involved on the project, Brandon Sookraj, on his blog on his blog (https://sookrajrobotics.wordpress.com).

**Funding and support**

This project is supported by the National Science Foundation grant “III: Small: Distributed Semantic Information Processing Applied to Camera Sensor Networks” (Award number 1717656).

*Disclaimer: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.*

## Vision-based formation control

The goal of formation control is to move a group of agents in order to achieve and maintain a set of desired relative positions. This problem has a long history, and latest trends emphasize the use of vision-based solution. In this setting, the measurement of the relative direction (i.e., bearing) between two agents can be quite accurate, while the measurement of their distance is typically less reliable.

We propose a general solution which is based on pure bearing measurements, optionally augmented with the corresponding distances. As opposed to the state of the art, our control law does not require auxiliary distance measurements or estimators, it can be applied to leaderless or leader-based formations with arbitrary topologies. Our framework is based on distributed optimization, and it has global convergence guarantees.

We have experimentally validated our approach on a platform of three quadrotors.

## The space of essential matrices as a Riemannian manifold

The images of 3-D points in two views are related by the so-called _essential matrix_.

There have been attempts to characterize the space of valid essential matrices as a Riemannian manifold. These approaches either put an unnatural emphasis on one of the two cameras, or do not accurately take into account the geometric meaning of the representation.

We addressed these limitations[^1] by proposing a new parametrization which aligns the global reference frame with the baseline between the two cameras. This provides a symmetric, geometrically meaningful representation which can be naturally derived as a quotient manifold. This not only provides a principled way to define distances between essential matrices, but it also sheds new light on older results (such as the well-known twisted pair ambiguity).

We provide an implementation of the basic function for working with the essential manifold integrated with the Matlab toolbox MANOPT. Download link: Manopt 1.06b with essential manifold.

## Distributed localization algorithms

Imagine a wireless camera network, where each camera has a piece of local information, e.g., the pose of the object from a specific viewpoint or the relative poses with respect to the neighboring cameras.

It is natural to look for distributed algorithms which merge all these local measurements into a single, globally consistent estimate. I derived such algorithms by formulating a global optimization problem over the space of poses, and shown their convergence from a large set of initial conditions using the aforementioned theoretical tools.

## Consensus algorithms on Riemannian manifolds

Given a group of agents which move in Euclidean space and communicate according to a given communication graph, standard consensus algorithms provide a protocol which, as time passes, brings all the agents to a common location. The key aspect here is that only local communications are used. These algorithms, however, do not apply when the agents evolve on a manifold (for instance, imagine a group of satellites synchronizing their poses). Using my theoretical work, I proposed a natural extension for this case, and characterized its convergence for a large class of manifolds.

This work was awarded Best Student Paper and Best Student Paper Runner-up at the IEEE Conference for Decision and Control (CDC) in 2012 and 2011, respectively.

## Distributed optimization on Riemannian manifolds

I worked on distributed optimization problems involving variables lying on non-linear spaces (that is, Riemannian manifolds) using extensions of gradient descent algorithms with fixed step size. I developed novel theoretical tools which significantly broadened the state of the art for determining sufficient conditions for global behaviors (algorithm convergence) using only local information. These tools have been used in consensus algorithms, camera localization and formation control.

## Motion segmentation

My initial research included the comparison of different algorithms for segmenting multiple moving objects in a monocular video. For this purpose,

I created the Hopkins 155 dataset, which, since its introduction, has been used in over 150 scholarly articles and is a de-facto standard benchmark in this field.

The following is a frame from the dataset, together with the manually labelled feature tracks.

Please refer to the dataset page on the JHU Vision Lab for more detailed information and download instructions.