Yes, you heard it right! We’re not just talking about playing games or solving puzzles; we’re creating intelligent beings that can build stuff on their own. And let me tell you, this is no easy feat.

To kick things off, why would anyone want to create a building agent in Minecraft? Well, for starters, it’s an open-world game with endless possibilities. You could have your agents construct anything from simple houses to massive castles or even entire cities! And that’s not all the beauty of this game is its creativity and freedom. Your agents can explore the world freely and achieve different kinds of goals without any restrictions on how they do it.

But building an agent in Minecraft isn’t a walk in the park, my friend. It requires a lot of training data to teach them what blocks go where and how to use tools effectively. And that’s just scratching the surface you also need to incorporate explicit knowledge representation and reasoning into your agents for more complex tasks.

Now, let me introduce our framework for building agents in Minecraft: OpenCog Hyperon. This platform is designed specifically for coordinating numerous AI services deployed on a SingularityNET ecosystem. And that’s not all it can be applied to various other applications as well! We’re currently using this framework to create open-world Minecraft tasks, such as finding caves or building waterfalls.

But let me tell you about our ultimate goal: creating an AGI (Artificial General Intelligence) that can coordinate skills of agents deployed on the platform. This requires a language for AI services and a framework applicable to coordination of numerous AI services. And that’s where we come in! We’re developing OpenCog Hyperon as such a platform, which is why our Minecraft agent capabilities are just one part of this larger picture.

So what kind of tasks can our agents perform? Well, let me introduce you to our benchmark for creative agents: the Building Creation task. This involves following language instructions to construct buildings in Minecraft based on a set of diverse prompts. For example, we might ask our agent to build “a huge Minecraft volcano built of ice.” And guess what? Our agents can do it!

But let’s not forget about the metrics for evaluating these open-ended building creation tasks. We have human evaluators and a novel evaluator based on GPT-4V, which is pretty cool if you ask me. And we’re constantly improving our framework to make it more efficient and effective at creating intelligent agents in Minecraft.

Later!