Diffusion models have become the prevailing approach for generating videos. Yet, their dependence on large-scale web data, which varies in quality, frequently leads to outcomes lacking visual appeal and not aligning well with the provided textual prompts. Despite advancements in recent times, there is still room for enhancing the visual quality of generated videos. One…

Transformer models are crucial in machine learning for language and vision processing tasks. Transformers, renowned for their effectiveness in sequential data handling, play a pivotal role in natural language processing and computer vision. They are designed to process input data in parallel, making them highly efficient for large datasets. Regardless, traditional Transformer architectures must improve…

The use of diffusion models for interactive image generation is a burgeoning area of research. These models are lauded for creating high-quality images from various prompts and finding applications in digital art, virtual reality, and augmented reality. However, their real-time interaction capabilities are limited, particularly in dynamic environments like the Metaverse and video game graphics. …

Researchers from Alibaba, Zhejiang University, and Huazhong University of Science and Technology have come together and introduced a groundbreaking video synthesis model, I2VGen-XL, addressing key challenges in semantic accuracy, clarity, and spatio-temporal continuity. Video generation is often hindered by the scarcity of well-aligned text-video data and the complex structure of videos. To overcome these obstacles,…

Although it would be helpful for applications like autonomous driving and mobile robotics, monocular estimation of metric depth in general situations has been difficult to achieve. Indoor and outdoor datasets have drastically different RGB and depth distributions, which presents a challenge. Another issue is the inherent scale ambiguity in photos caused by not knowing the…

In response to the challenging task of generating realistic 3D human-object interactions (HOIs) guided by textual prompts, researchers from Northeastern University, Hangzhou Dianzi University, Stability AI, and Google Research have introduced an innovative solution called HOI-Diff. The intricacies of human-object interactions in computer vision and artificial intelligence have posed a significant hurdle for synthesis tasks.…

LLMs have ushered in a new era of general-purpose vision systems, showcasing their prowess in processing visual inputs. This integration has led to the unification of diverse vision-language tasks through instruction tuning, marking a significant stride in the convergence of natural language understanding and visual perception. Researchers from Johns Hopkins University, Meta, University of Toronto,…