Enlarge / MineDojo’s AI Can perform complex tasks Minecraft.


A paper describing MineDojo, Nvidia’s generalist AI An agent who can act on written prompts Minecraft, won The 2022 Outstanding Datasets & Benchmarks Paper Award NeurIPS (Neural Information Processing Systems) conference, Nvidia On Monday, the results were revealed.

To play the MineDojo framework, you must train it. MinecraftResearchers fed it 730,000 Minecraft YouTube videos (with over 2.2 billion words transcribed), 7,000 scraped pages from the Minecraft wiki, 340,000 Reddit posts, and 6.6 Million Reddit comments Minecraft gameplay.

The researchers used this data to create a customized transformer model called MineCLIP, which associates video clips and specific in-game items. Minecraft activities. A MineDojo agent can be told what to do by someone using high-level natural languages such as “find a desert pyramid” Or “build a nether portal and enter it,” MineDojo will perform the necessary steps to make it all happen in the game.

Examples of tasks that MineDojo can perform.

Here are some examples of tasks MineDojo can do.


MineDojo is a flexible agent that can apply learned actions to other behaviors in the game. As Nvidia writes, “While researchers have long trained autonomous AI agents in video-game environments such as StarCraft, Dota, and Go, these agents are usually specialists in only a few tasks. So Nvidia researchers turned to Minecraft, the world’s most popular game, to develop a scalable training framework for a generalist agent—one that can successfully execute a wide variety of open-ended tasks.”

ezgif.com gif maker


The award-winning paper, “MINEDOJO: Building Open-Ended Embodied Agents with Internet-Scale Knowledge,” debuted It was published in June. Linxi Fan is the author. Nvidia Guanzhi Zhang, Yunfan Jinang, Ajay Mandalkar, Yuncong Yang and Haoyi Zhu.

On its website, you can see MineDojo in action. official websiteThe code is for MineDojo And MineCLIP It is also available on GitHub.