Start state
Ensemble Latent Space Roadmap for Improved Robustness in Visual Action Planning
Martina Lippi*, Michael C. Welle*, Andrea Gasparri, and Danica Kragic
Planning in learned latent spaces helps to decrease the dimensionality of raw observations. In this work, we propose to leverage the ensemble paradigm to enhance the robustness of latent planning systems. We rely on our Latent Space Roadmap (LSR) framework, which builds a graph in a learned structured latent space to perform planning. Given multiple LSR framework instances, that differ either on their latent spaces or on the parameters for constructing the graph, we use the action information as well as the embedded nodes of the produced plans to define similarity measures. These are then utilized to select the most promising plans. We validate the performance of our Ensemble LSR (ENS-LSR) on simulated box stacking and grape harvesting tasks as well as on a real-world robotic T-shirt folding experiment.
*Contributed equally and listed in alphabetical order
Download Preprint
Method
Our Method:
Folding execution videos
Fold 1
Goal state
Fold 2
Start state
Goal state
Fold 3
Start state
Goal state
Fold 4
Start state
Goal state
Fold 5
Start state
Goal state
HTM examples
Example of erroneous plans obtained with method from the paper Liu, Kara, et al. "Hallucinative topological memory for zero-shot visual planning." International Conference on Machine Learning, 2020Code Repository
Available after paper acceptance
Datasets
Available after paper acceptance