HTP Graphics

UNIQUENESS // While many control architectures have been designed to allow the generation of walking motions on quasi- flat terrains, the assumptions used usually fail when traversing more difficult terrains. Other approaches compute motions on those terrains but their complexity makes them too difficult to be used online. To close this gap, work has been undertaken to improved control algorithms with new machine learning techniques. While the two techniques may lay on the same principles, they yield different advantages that can be combined. In the Dynamic Robot Systems group, we explored many different ways to combine them: quickly generate motion data using control, guide search of reinforcement learning with the generated data, initialize control algorithms using learned solutions, improve robustness of controllers using reinforcement learning, automatically adjust controller parameters, etc. The goal is to build increasingly versatile and robust controllers but also to ease the development of such controllers to allow fast deployment. RAIN PROGRESS // We developed and tested a motion planner for the ANYmal quadruped robot. For this motion planner, the model used is a complex model that takes explicitly into account the dynamic and kinematic limits of our quadruped robots. At the expense of a relatively long computation time (computation time about as long as the planned trajectory duration), this algorithm has a very general formulation that allows it to generate motions on any type of terrains with an efficient behaviour. To be able to use this motion planner in more practical situations, we have investigated how to use the solution generated offline to speed up the online computation. For that, we learn the generated solutions using supervised learning, then online we use the trained neural network to initialise the planner. Using this technique, we have shown that the stability of the algorithm is greatly increased, and the online computation can be reduced by a factor of 5. FUTURE ASPIRATIONS // To continue improving the locomotion of quadruped robots, we are now working on combining the two approaches used so far, supervised learning and reinforcement. The combination will allow us to have very efficient locomotion on different terrains while having a very short computation time and therefore a very fast and reactive behaviour providing the ability to traverse difficult zones in a short time and therefore accessing large areas even with the limited battery duration of quadruped robots. This will take robots into places that wheeled robots can't access with sensors that are too heavy (like shielded radiation sensors) for Unmanned Aerial Vehicles (UAVs). Another aspect that still needs to be explored is how can we apply what we have learned for locomotion to manipulation. Having robots being able to assess the danger of a zone is great, but having a robot being able to reduce it is even better. Hence, robots need to be able to act on the environment. Adding a manipulator on the robot and using the diverse remote handling techniques developed inside RAIN will allow quadruped robots to be used as a "mobile glovebox". Another part of our work focused on discovering and improving the limits of the controllers using reinforcement learning. Each control algorithm is based on models that have a limited validity region (due to model approximations or imprecise parameters). Since the control architecture of quadruped robots are complex and are composed of many controllers/ planners using different frequencies and algorithms, discovering the limits of the system is very complex. Therefore, we have used reinforcement learning to learn the limits of the control architecture and extend them. Using this technique, we have shown that we were able to extend a controller only designed for quasi- flat terrains into a controller that can robustly walk on three-dimensional terrains and that is twice as robust to disturbances or error in the robot estimated mass. REMOTE INSPECTION 35

• Cover