Relevant Publications
2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Abstract: Reinforcement Learning (RL) has been widely used to solve tasks where the environment consistently provides a dense reward value. However, in real-world scenarios, rewards can often be poorly defined or sparse. Auxiliary signals are indispensable for discovering efficient exploration strategies and aiding the learning process. In this work, inspired by intrinsic motivation theory, we postulate that the intrinsic stimuli of novelty and surprise can assist in improving exploration in complex, sparsely rewarded environments. We introduce a novel sample-efficient method able to learn directly from pixels, an image-based extension of TD3 with an autoencoder called \textit{NaSA-TD3}. The experiments demonstrate that NaSA-TD3 is easy to train and an efficient method for tackling complex continuous-control robotic tasks, both in simulated environments and real-world settings. NaSA-TD3 outperforms existing state-of-the-art RL image-based methods in terms of final performance without requiring pre-trained models or human demonstrations.
Trajectory tracking control for multiple quadrotors based on a neurobiological-inspired system
2019 Third IEEE International Conference on Robotic Computing (IRC)
Abstract: Operating more than two quadrotors at the same time can be complicated and unsafe, for that reason, this paper presents a control system capable of autonomously operating multiple quadrotors simultaneously, efficiently and safe. The control system presented is inspired by the working principle of the mammalian's brain, where a mathematical model of the limbic system is implemented; This model, known as BELBIC, has as main tasks the stabilization of the quadrotors as well as the autonomous tracking of trajectories. In addition, a fuzzy system is presented in order to maintain a constant separation between quadrotors and keep a specific formation. Also, a leader-follower configuration is implemented, which greatly simplifies the operation of the quadrotors. The proposed control method is then verified through a set of tests, both in a real environment and simulations. The results demonstrated the effectiveness and satisfactory performance of the proposed method.
Comparison of Model-Based and Model-Free Reinforcement Learning for Real-World Dexterous Robotic Manipulation Tasks
2023 IEEE International Conference on Robotics and Automation (ICRA)
Abstract: Model-Free Reinforcement Learning (MFRL) has shown significant promise for learning dexterous robotic manipulation tasks, at least in simulation. However, the high number of samples, as well as the long training times, prevent MFRL from scaling to complex real-world tasks. Model-Based Reinforcement Learning (MBRL) emerges as a potential solution that, in theory, can improve the data efficiency of MFRL approaches. This could drastically reduce the training time of MFRL and increase the application of RL for real-world robotic tasks. This article presents a study on the feasibility of using the state-of-the-art MBRL to improve the training time for two real-world dexterous manipulation tasks. The evaluation is conducted on a real low-cost robot gripper where the predictive model and the control policy are learned from scratch. The results indicate that MBRL is capable of learning accurate models of the world but does not show clear improvements in learning the control policy in the real world as prior literature suggests should be expected.
Quadrotor obstacle detection and avoidance system using a monocular camera
2018 3rd Asia-Pacific Conference on Intelligent Robot Systems (ACIRS)
Abstract: The ability to detect and avoid obstacles is a fundamental characteristic in autonomous aerial vehicles systems. Most conventional methods to detect obstacles using a combination of multiples sensors and cameras, but a quadrotor has the great limitation of the payload that can carry so the number of sensors onboard are extremely limited. To cope with this problem, this paper presents a novel obstacle detection method based on salient object detection and rasterization image using only a single camera built-in a cheap quadrotor. The video of the quadrotor camera is sent to the ground station via a wireless connection, where a salient map is generated using a histogram back-projection algorithm, this map is divided into Nx × Ny squares, finally calculating and comparing the intensity of each square with a threshold, we can detect and locate a possible obstacle. We demonstrate and evaluate successfully the proposed method with a parrot quadrotor, navigate it through different scenarios. Good performance on the experiments supports the proposed method, it can detect obstacles without any preprocessing data.
Theses
Enhancing Reinforcement Learning Efficiency and Performance: Analysis, Challenges, and Human-Inspired Solutions
PhD Thesis. The University of Auckland , New Zealand, 2024
Neurobiological-Inspired Control of Multiple Quadrotors Using Brain Emotional Learning-Based Intelligent Controller (BELBIC) and Fuzzy Logic
Master Thesis. Kyung Hee University, South Korea, 2019
Diseño e implementación de un prototipo de robot asistente para personas con discapacidad motriz y adultos mayores, basado en inteligencia artificial
Engineer Thesis, Electronic Engineering. Salesian Polytechnic University 2014
​​
Image-Based Deep Reinforcement Learning with Intrinsically Motivated Stimuli: On the Execution of Complex Robotic Tasks2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024
​
CTD4-A Deep Continuous Distributional Actor-Critic Agent with a Kalman Fusion of Multiple Critics
arXiv preprint arXiv:2405.02576, 2024​
​
Racing Towards Reinforcement Learning based control of an Autonomous Formula SAE Car
arXiv preprint arXiv:2308.13088, 2023​
​
Comparison of model-based and model-free reinforcement learning for real-world dexterous robotic manipulation tasks
2023 IEEE International Conference on Robotics and Automation (ICRA), 871-878, 2023​
​
Action-Conditioned Frame Prediction Without Discriminator
International Conference on Machine Learning, Optimization, and Data Science, 2021
​
​Trajectory tracking control for multiple quadrotors based on a neurobiological-inspired system
2019 Third IEEE International Conference on Robotic Computing (IRC), 465-470, 2019​
​
Quadrotor obstacle detection and avoidance system using a monocular camera
2018 3rd Asia-Pacific Conference on Intelligent Robot Systems (ACIRS), 78-81, 2018​
​
Hand gestures recognition using machine learning for control of multiple quadrotors
2018 IEEE Sensors Applications Symposium (SAS), 1-6, 2018​
​
An integrated system for gait analysis using FSRs and an IMU
2018 Second IEEE International Conference on Robotic Computing (IRC), 347-351, 2018​
​
Multiple quadrotors flight formation control based on sliding mode control and trajectory tracking
2018 International Conference on Electronics, Information, and Communication, 2018​
​
Analysis and determination of minimum requirements for a data link communication system for unmanned aerial vehicles-UAV's
2016 IEEE Ecuador Technical Chapters Meeting (ETCM), 1-6, 2016​
​
Analysis, Design, and Implementation of an Autopilot for Unmanned Aircraft-UAV's Based on Fuzzy Logic
2015 Asia-Pacific Conference on Computer-Aided System Engineering, 196-201, 2015​
​
SA3M: An interactive robot to provide support for the elderly
2014 IEEE International Autumn Meeting on Power, Electronics and Computing, 2014​
​
Fuzzy controller for automatic microphone gain control in an autonomous support system for elderly
2014 IEEE 16th International Conference on e-Health Networking, Applications, 2014​​
​
Analysis of movements during the process of the march to ascend stairs by means of sensor Kinect
2013 Pan American Health Care Exchanges (PAHCE), 2013