Multimodal

VIMA
Multimodal LLM for robot manipulation; unifies diverse robotics tasks in a single prompting framework.
Website Paper Code Dataset Announcement
VIMA
VIMA: General Robot Manipulation with Multimodal Prompts
✨ Oral Presentation ✨. 1st NeurIPS Workshop on Foundation Models for Decision Making (FMDM), 2022
Yunfan Jiang, Agrim Gupta, Zichen Zhang, Guanzhi Wang, Yongqiang Dou, Yanjun Chen, Li Fei-Fei, Anima Anandkumar, Yuke Zhu, Linxi "Jim" Fan
Cite Project PDF Arxiv Twitter FMDM Workshop Code Models Benchmark Dataset
VIMA: General Robot Manipulation with Multimodal Prompts
MineDojo
✨NeurIPS Outstanding Paper Award✨. Large-scale open-ended agent learning framework in Minecraft.
Website Paper Code Database Doc Blog Announcement Intro
MineDojo
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge
💫✨ Outstanding Paper Award ✨💫. Neural Information Processing Systems (NeurIPS) Dataset & Benchmark, 2022
Linxi "Jim" Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, Anima Anandkumar
Cite Project PDF Arxiv Twitter NeurIPS 2022 Code Models Database Blog TwoMinutePaper Video NeurIPS Interview MIT Tech Review Ars Technica
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge
Training and Deploying Visual Agents at Scale
My Stanford Ph.D. thesis advised by Fei-Fei Li. Learn the rules like a pro, so you can break them like an artist — Picasso.
Sep 27, 2021
PDF SECANT iGibson SURREAL RubiksNet
Training and Deploying Visual Agents at Scale
Training and Deploying Visual Agents at Scale
My Stanford Ph.D. Thesis advised by Prof. Fei-Fei Li. 132 pages in total, enjoy!
Linxi "Jim" Fan
Cite PDF Stanford Library
Training and Deploying Visual Agents at Scale
iGibson 1.0: A Simulation Environment for Interactive Tasks in Large Realistic Scenes
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021
Bokui Shen, Fei Xia, Chengshu Li, Roberto Martín-Martín, Linxi "Jim" Fan, Guanzhi Wang, Claudia Pérez-D'Arpino, Shyamal Buch, Sanjana Srivastava, Lyne P. Tchapmi, Micael E. Tchapmi, Kent Vainio, Josiah Wong, Li Fei-Fei, Silvio Savarese
Cite Project PDF Arxiv IROS 2021 Code
iGibson 1.0: A Simulation Environment for Interactive Tasks in Large Realistic Scenes
World of Bits: An Open-Domain Platform for Web-Based Agents
International Conference on Machine Learning (ICML), 2017
Tianlin Shi, Andrej Karpathy, Linxi "Jim" Fan, Jonathan Hernandez, Percy Liang
Cite Project PDF ICML 2017 Code Video
World of Bits: An Open-Domain Platform for Web-Based Agents