Since it enables humans to teach robots any skill, imitation learning via human-provided demonstrations is a promising approach for creating generalist robots. Lane-following in mobile robots, basic pick-and-place manipulation, and more delicate manipulations like spreading pizza sauce or inserting a battery may all be taught to robots through direct behavior cloning. However, rather than merely…

Robotics is currently exploring how to enhance complex control tasks, such as manipulating objects or handling deformable materials. This research niche is crucial as it promises to bridge the gap between current robotic capabilities and the nuanced dexterity found in human actions. A central challenge in this area is developing models that can accurately indicate…

The problem of achieving superior performance in robotic task planning has been addressed by researchers from Tsinghua University, Shanghai Artificial Intelligence Laboratory, and Shanghai Qi Zhi Institute by introducing Vision-Language Planning (VILA). VILA integrates vision and language understanding, using GPT-4V to encode profound semantic knowledge and solve complex planning problems, even in zero-shot scenarios. This…

The team of researchers from NYU and Meta aimed to address the challenge of robotic manipulation learning in domestic environments by introducing DobbE, a highly adaptable system capable of learning and adapting from user demonstrations. The experiments demonstrated the system’s efficiency while highlighting the unique challenges in real-world settings. The study recognizes recent strides in…

In this paper, researchers have introduced a NeRF-based mapping method called H2-Mapping, aimed at addressing the need for high-quality, dense maps in real-time applications, such as robotics, AR/VR, and digital twins. The key problem they tackle is the efficient generation of detailed maps in real-time, particularly on edge computers with limited computational power. They highlight…

With the increase in the popularity and use cases of Artificial Intelligence, Imitation learning (IL) has shown to be a successful technique for teaching neural network-based visuomotor strategies to perform intricate manipulation tasks. The problem of building robots that can do a wide variety of manipulation tasks has long plagued the robotics community. Robots face…

Teaching robots complicated manipulation skills through observation of human demonstrations has shown promising results. Providing extensive manipulation demonstrations is time-consuming and labor costly, making it challenging to scale up this paradigm to real-world long-horizon operations. However, not all facets of a task are created equal. A new study by NVIDIA and Georgia Institute of Technology…

In many domains that involve machine learning, a widely successful paradigm for learning task-specific models is to first pre-train a general-purpose model from an existing diverse prior dataset and then adapt the model with a small addition of task-specific data. This paradigm is attractive to real-world robot learning since collecting data on a robot is…

Training robots to perform various manipulation behaviors has been made possible by imitation learning from human demonstrations. One popular method involves having human operators teleoperate with robot arms through various control interfaces, producing multiple demonstrations of robots performing different manipulation tasks, and then using the data to train the robots to perform these tasks independently.…

In robotics, researchers face challenges in using reinforcement learning (RL) to teach robots new skills, as these skills can be sensitive to changes in the environment and robot structure. Current methods need help generalizing to new combinations of robots and tasks and handling complex, real-world tasks due to architectural complexity and strong regularisation. To tackle…