Microsoft’s recent development that allows robots to interact with the real world using ChatGPT technology is a significant step forward in AI and robotics.
This development can revolutionize how we interact with robots, making them more intuitive and responsive to our needs. By enabling robots to understand natural language, we can communicate with them more effectively and efficiently.
Researchers from Microsoft recently revealed an experimental structure for utilizing the verbal aptitudes of ChatGPT, a celebrated AI language model developed by OpenAI, to regulate robots and drones.
ChatGPT can generate code that can direct the movements of robots through natural language commands. A human then inspects the output and tweaks it until the task is accomplished effectively.
A paper called “ChatGPT for Robotics: Design Principles and Model Abilities,” by Sai Vemprala, Rogerio Bonatti, Arthur Bucker, and Ashish Kapoor of Microsoft’s Autonomous Systems and Robotics Group presented the findings of the study.
In a demonstration video, Microsoft demonstrated using robots with code written by ChatGPT to follow human instructions. This includes using a robot arm to assemble blocks into a formation of the Microsoft logo, flying a drone to inspect shelves, and having a robot with vision capabilities locate objects.
The researchers enabled ChatGPT to interact with robotics by teaching it a tailored robotics API. With commands like “pick up the ball,” ChatGPT can write robotics control code in the same manner as penning poetry or writing an essay.
Once the code has been checked and corrected by a person for correctness and security, it can be carried out by the person in charge, and its achievement assessed.
ChatGPT is not a system that works independently. Instead, it enhances humans’ capabilities in programming robotic control. According to the paper, ChatGPT “acts as a tool to augment human capacity” rather than completely automating the process.
Feedback for the performance of ChatGPT has been sourced from humans, predominantly as text-based comments. Additionally, the researchers have seen positive results when they input visual information into ChatGPT.
ChatGPT was tasked with commanding a robot to catch a basketball, with feedback from a camera. The researchers enabled it to estimate both the ball’s appearance and the sky in the image provided by the camera using SVG code.
The LLM may be capable of constructing and tracking an internal representation of reality beyond simply analyzing text-based probabilities.
Applying the most current technology, large language models, to robotic control has yielded rudimentary results. Nonetheless, Microsoft believes that with a ChatGPT interface, robotics could be accessible to a much larger group of people.
This study aims to determine if ChatGPT can go beyond text and make inferences about the physical world to assist with robotic activities, as articulated in a Microsoft Research blog post. We aim to facilitate people’s interactions with robots, eliminating the necessity for them to learn intricate programming languages or knowledge about robotic systems.
Microsoft’s development that allows robots to interact with the real world using ChatGPT technology is a significant breakthrough in AI and robotics. We can create more intuitive and responsive machines by enabling robots to understand natural language.
However, it is important to continue to address ethical concerns and ensure that robots are developed and used responsibly and ethically. With the right approach, this technology can continue to drive innovation and improve our lives.
Source: Ars Technica
