For example, there's a technique called domain randomization, and how it works is that you train a robot in a simulation, but in parallel in . different simulations, each with different physical parameters like gravity, friction, weight, etc. It's literally . different worlds. If we had an agent that could master all of these . different configurations of reality, then our actual physical world would just be .. a simulation. In this case, we are able to generalize directly from the virtual world to the real world. This is actually what we did in the Eureka sequel. We train the agent using different random simulations, and then transfer it directly to the real world without further fine-tuning. I believe this approach is efficient. If we have different virtual worlds (including game worlds) and an agent capable of mastering different skills in all of those worlds, then the real world is just a part of a larger distribution.
Dr. Eureka's project with everyone? Jim Fan Sure. In Dr. Eureka's project, we ghana phone numbers continue to use LLM as a robot programmer based on the Eureka results. LLM writes code that specifies the simulation parameters, such as the domain randomization parameters. After a few iterations, the policy we trained in the simulation generalized to the real world. One specific example we showed was that we had a robot dog that walked on a yoga ball and was able to not only maintain balance but also walk forward. There is a very interesting comment from someone who tried this task with his real dog and found that his dog could do it. So, our neural network is somehow outperforming the performance of "real dogs". Sonia Huang I'm pretty sure my dog can't do that, haha. Jim Fan Yes, artificial dog intelligence (ADI), that's the topic of the next adventure book.
Sonia Huang In the field of virtual worlds, there have been a lot of amazing 3D models and video generation recently, many of which are based on Transformers. Do you think we have reached a stage where we can achieve our desired goals with these architectures? Or do you think that some breakthroughs in model architecture are still needed? Jim Fan Yes, I don't think we've reached the full limits of the Transformer architecture in terms of the basic robot model. The bigger bottleneck right now is the data issues.As I mentioned earlier, we can’t just grab the data that the robot controls from the internet. We have to collect this data in simulations or with real robots. Once we have a mature data pipeline, we can tokenize this data and feed it into Transformer for compression, just like Transformer predicts the next word on Wikipedia.