Relevant Publications
2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Abstract: Reinforcement Learning (RL) has been widely used to solve tasks where the environment consistently provides a dense reward value. However, in real-world scenarios, rewards can often be poorly defined or sparse. Auxiliary signals are indispensable for discovering efficient exploration strategies and aiding the learning process. In this work, inspired by intrinsic motivation theory, we postulate that the intrinsic stimuli of novelty and surprise can assist in improving exploration in complex, sparsely rewarded environments. We introduce a novel sample-efficient method able to learn directly from pixels, an image-based extension of TD3 with an autoencoder called \textit{NaSA-TD3}. The experiments demonstrate that NaSA-TD3 is easy to train and an efficient method for tackling complex continuous-control robotic tasks, both in simulated environments and real-world settings. NaSA-TD3 outperforms existing state-of-the-art RL image-based methods in terms of final performance without requiring pre-trained models or human demonstrations.
CTD4 - A Deep Continuous Distributional Actor-Critic Agent with a Kalman
Fusion of Multiple Critics
2025 The 39th Annual AAAI Conference on Artificial Intelligence (AAAI)
Abstract: Categorical Distributional Reinforcement Learning (CDRL) has demonstrated superior sample efficiency in learning complex tasks compared to conventional Reinforcement Learning (RL) approaches. However, the practical application of CDRL is encumbered by challenging projection steps, detailed parameter tuning, and domain knowledge. This paper addresses these challenges by introducing a pioneering Continuous Distributional Model-Free RL algorithm tailored for continuous action spaces. The proposed algorithm simplifies the implementation of distributional RL, adopting an actor-critic architecture wherein the critic outputs a continuous probability distribution. Additionally, we propose an ensemble of multiple critics fused through a Kalman fusion mechanism to mitigate overestimation bias. Through a series of experiments, we validate that our proposed method provides a sample-efficient solution for executing complex continuous-control tasks.
Comparison of Model-Based and Model-Free Reinforcement Learning for Real-World Dexterous Robotic Manipulation Tasks
2023 IEEE International Conference on Robotics and Automation (ICRA)
Abstract: Model-Free Reinforcement Learning (MFRL) has shown significant promise for learning dexterous robotic manipulation tasks, at least in simulation. However, the high number of samples, as well as the long training times, prevent MFRL from scaling to complex real-world tasks. Model-Based Reinforcement Learning (MBRL) emerges as a potential solution that, in theory, can improve the data efficiency of MFRL approaches. This could drastically reduce the training time of MFRL and increase the application of RL for real-world robotic tasks. This article presents a study on the feasibility of using the state-of-the-art MBRL to improve the training time for two real-world dexterous manipulation tasks. The evaluation is conducted on a real low-cost robot gripper where the predictive model and the control policy are learned from scratch. The results indicate that MBRL is capable of learning accurate models of the world but does not show clear improvements in learning the control policy in the real world as prior literature suggests should be expected.
Quadrotor obstacle detection and avoidance system using a monocular camera
2018 3rd Asia-Pacific Conference on Intelligent Robot Systems (ACIRS)
Abstract: The ability to detect and avoid obstacles is a fundamental characteristic in autonomous aerial vehicles systems. Most conventional methods to detect obstacles using a combination of multiples sensors and cameras, but a quadrotor has the great limitation of the payload that can carry so the number of sensors onboard are extremely limited. To cope with this problem, this paper presents a novel obstacle detection method based on salient object detection and rasterization image using only a single camera built-in a cheap quadrotor. The video of the quadrotor camera is sent to the ground station via a wireless connection, where a salient map is generated using a histogram back-projection algorithm, this map is divided into Nx × Ny squares, finally calculating and comparing the intensity of each square with a threshold, we can detect and locate a possible obstacle. We demonstrate and evaluate successfully the proposed method with a parrot quadrotor, navigate it through different scenarios. Good performance on the experiments supports the proposed method, it can detect obstacles without any preprocessing data.
Theses
Enhancing Reinforcement Learning Efficiency and Performance: Analysis, Challenges, and Human-Inspired Solutions
PhD Thesis. The University of Auckland , New Zealand, 2024
Neurobiological-Inspired Control of Multiple Quadrotors Using Brain Emotional Learning-Based Intelligent Controller (BELBIC) and Fuzzy Logic
Master Thesis. Kyung Hee University, South Korea, 2019
Diseño e implementación de un prototipo de robot asistente para personas con discapacidad motriz y adultos mayores, basado en inteligencia artificial
Engineer Thesis, Electronic Engineering. Salesian Polytechnic University 2014