There has been a recent uptick in the development of general-purpose multimodal AI assistants capable of following visual and written directions, thanks to the remarkable success of Large Language Models (LLMs). By utilizing the impressive reasoning capabilities of LLMs and information found in huge alignment corpus (such as image-text pairs), they demonstrate the immense potential…

The intersection of artificial intelligence and creativity has witnessed an exceptional breakthrough in the form of text-to-image (T2I) diffusion models. These models, which convert textual descriptions into visually compelling images, have broadened the horizons of digital art, content creation, and more. Yet this rapidly evolving area of Personalized T2I generation study grapples with several core…

Deep learning has revolutionized view synthesis in computer vision, offering diverse approaches like NeRF and end-to-end style architectures. Traditionally, 3D modeling methods like voxels, point clouds, or meshes were employed. NeRF-based techniques implicitly represent 3D scenes using MLPs. Recent advancements focus on image-to-image approaches, generating novel views from collections of scene images. These methods often…

MoD-SLAM is a state-of-the-art method for Simultaneous Localization And Mapping (SLAM) systems. In SLAM systems, it is challenging to achieve real-time, accurate, and scalable dense mapping. To address these challenges, researchers have introduced a novel method focusing on unbounded scenes using only RGB images. Existing neural SLAM methods often rely on RGB-D input which leads…

The task of view synthesis is essential in both computer vision and graphics, enabling the re-rendering of scenes from various viewpoints akin to the human eye. This capability is vital for everyday tasks and fosters creativity by allowing the envisioning and crafting of immersive objects with depth and perspective. Researchers at Dyson Robotics Lab aim…

Integrating natural language understanding with image perception has led to the development of large vision language models (LVLMs), which showcase remarkable reasoning capabilities. Despite their progress, LVLMs often encounter challenges in accurately anchoring generated text to visual inputs, manifesting as inaccuracies like hallucinations of non-existent scene elements or misinterpretations of object attributes and relationships. Researchers…

The landscape of image segmentation has been profoundly transformed by the introduction of the Segment Anything Model (SAM), a paradigm known for its remarkable zero-shot segmentation capability. SAM’s deployment across a wide array of applications, from augmented reality to data annotation, underscores its utility. However, SAM’s computational intensity, particularly its image encoder’s demand of 2973…

In artificial intelligence, integrating multimodal inputs for video reasoning stands as a frontier, challenging yet ripe with potential. Researchers increasingly focus on leveraging diverse data types – from visual frames and audio snippets to more complex 3D point clouds – to enrich AI’s understanding and interpretation of the world. This endeavor aims to mimic human…

The progress and development of artificial intelligence (AI) heavily rely on human evaluation, guidance, and expertise. In computer vision, convolutional networks acquire a semantic understanding of images through extensive labeling provided by experts, such as delineating object boundaries in datasets like COCO or categorizing images in ImageNet.  Similarly, in robotics, reinforcement learning often relies on…

Artificial intelligence has significantly advanced in developing systems that can interpret and respond to multimodal data. At the forefront of this innovation is Lumos, a groundbreaking multimodal question-answering system designed by researchers at Meta Reality Labs. Unlike traditional systems, Lumos distinguishes itself by its exceptional ability to extract and understand text from images, enhancing the…