Projects & Awards

Best Paper Awards

  • Yasmeen Abu-Kheil, Gastone Ciuti, Marco Mura, Jorge Dias, Paolo Dario, Lakmal Seneviratne, “Vision/Inertial-Based Image Mapping for Capsule Endoscopy”, in UAR Graduate Students Research Conference, Abu Dhabi, UAE 2015.
  • Santos, Luís, Christophoros Christophorou, Eleni Christodoulou, Jorge Dias, and George Samaras. “On the development strategy of an architecture for e-health service robots.” In 6th International Conference on e-Health. 2014.
  • H. Aliakbarpour, J. Dias – “IMU-aided 3D Reconstruction based on Multiple Virtual Planes.” – DICTA’10, International Conference on Digital Image Computing: Techniques and Applications – Australian Pattern Recognition and Computer Vision Society Conference, IEEE Computer Society Press, 1-3 December 2010, Sydney, Australia.
  • Rett, J., Dias, J.: Autonomous Robot Navigation – A study using Optical Flow and log-polar image representation. In: Proceedings of the Colloquium of Automation, Salzhausen 2003/2004. (2004)

Active Projects


HANDLE – Developmental Pathway towards Autonomy and Dexterity in Robot In-Hand Manipulation

Timespan: 02/2009-02/2013



The HANDLE project aims at understanding how humans perform the manipulation of objects in order to replicate grasping and skilled in-hand movements with an anthropomorphic artificial hand, and thereby move robot grippers from current best practice towards more autonomous, natural and effective articulated hands. This project will focus on technological developments and, in addition, research fundamental multidisciplinary research aspects in order to endow the proposed robotic hand with advanced perception capabilities, high level feedback control and elements of intelligence that allow recognition of objects and context, reasoning about actions and a high degree of recovery from failure during the execution of dexterous tasks.


PROMETHEUS – Prediction and interpretation of human behaviour based on probabilistic structures and heterogeneous sensors

Timespan: 08/2008-08/2011


Description: The project intends to establish a link between fundamental sensing tasks and automated cognition
processes that concern the understanding a short-term prediction of human behaviour as well as
complex human interaction. The analysis of human behaviour is unrestricted environments, including
localization and tracking of multiple people and recognition of their activities, currently constitutes a
topic of intensive research in the signal processing and computer vision communities. This research is
driven by different important applications, including unattended surveillance and intelligent space


IRPS – Intelligent Robotic Porter System

Timespan: 01/2007-01/2011


Description: IRPS – Intelligent Robotic Porter System – is an Information Society Technologies
Specific Targeted Research Project.

Its objectives are to develop and integrate an accurate 3D sensing system, a LIDAR Imaging and Modular System (LIMS) as a modular component for robotics platforms, enabling fast dense mapping of large areas, populated with sparse objects; and to develop and demonstrate the LIMS in demanding application: an Intelligent Robotic Porter System to help porting & guiding the public in airport areas.


BACS – Bayesian Approach to Cognitive Systems

Timespan: 01/2006-02/2010


Description: BACS – Bayesian Approach to Cognitive Systems – is an Integrated Project conducted under the Thematic Priority: Information Society Technologies – Sub-topic: Cognitive Systems – of the 6th Framework Program of the European Commission.

In this project we investigate and apply Bayesian models and approaches in order to develop artificial cognitive systems that can carry out complex tasks in real world environments. We will take inspiration from the brains of mammals including humans and apply our findings to the developments of cognitive systems.



Timespan: 02/2015-01/2018


Description: GrowMeUp’s main aim is to increase the years of independent and active living, and the quality of life of older persons (age of 65+) with light physical or mental health problems who live alone at home and can find pleasure and relief in getting support or stimulation to carry out their daily activities over the ageing process. GrowMeUp will provide an affordable service robotic system able to learn the older persons needs and habits over time and enhance (‘grow up’/scale up) its functionality to compensate for the elder’s degradation of abilities, to support, encourage and engage the older persons to stay longer active, independent and socially involved, in carrying out their daily life at home. State of the art cloud computing technologies and machine learning mechanisms will be used, enabling the GrowMeUp robot to extend and increase its knowledge continuously over time. Robots will share and distribute their knowledge through the cloud, so that other “connected” robots can learn from each other’s experience, increasing thus their functionality/competencies and simultaneously reduce learning effort. Implicit daily activities support will be provided in a human like way characterized by behaviour and emotional understanding, intelligent dialoguing and personalized services provision. GrowMeUp will be introduced early enough to the elder person, creating thus a positive long-term relationship between the elder and the robot, considering persons as active collaborators with whom the robot can interact, so as to increase its knowledge about their personalized needs. Furthermore, the robot will be connected to a virtual care network that will provide for continuous care, but also motivation and education to the older persons of how to best use the platform. A group of relevant stakeholders will use the system over a nine month pilot period. The goal is to achieve global leadership in advanced solutions supporting active and healthy ageing.


BAMBI— Bottom-up Approaches to Machines dedicated to Bayesian Inference



Project number: 618024

Funding scheme: Collaborative project


Social Robot



The main goal of the proposed project is to provide an answer to the demographic change challenge, through knowledge transfer and the creation of strategic synergies between the project’s participating academia and industry regarding the development of an integrated Social Robotics system (SocialRobot) for “Ageing Well”.

The work focuses on bringing together the Robotic and Computer Science fields by integrating state of the art Robotic and Virtual Social Care Communities technologies and services to provide solutions to key issues of relevance for improved independent living and quality of life of elderly people and efficiency of care.

The SocialRobot development will be based on a “human centred approach” in which the elderly individual needs and requirements are met. The project will give the opportunity to participating SMEs with excellent credit in their domain and peripheral European regions, to reach excellence and compete with innovative products in the elderly care market, at European and International level.

The major challenges to be addressed in the project include the adaptation of state of the art robotic mobile platforms and their integration with a virtual collaborative social network to provide:

  1. Detection of individual needs and requirements related to ageing (e.g. physical mobility limitations or/and cognitive decline), and provision of support through timely involvement of care teams, consisting of different groups of people (family members, neighbours, friends) that collaborate dynamically and virtually; means independently of time and their physical locations; behaviour analysis to adapt social relationships and contexts of the elderly people as they age;
  2. Navigate indoors and unstructured environments and provide affective and empathetic user-robotic interaction, taking into account the capabilities of and acceptance by elderly users.

CHOPIN – Cooperation between Human and rObotic teams in catastroPhic INcidents



CHOPIN is a R&D project of the Mobile Robotics Lab of the Institute of Systems and Robotics at University of Coimbra, in Portugal. The acronym stands for Cooperation between Human and rObotic teams in catastroPhic INcidents. The project is funded by COMPETE program from “Fundação para a Ciência e a Tecnologia” (Portuguese Government), under the contract number FCOMP-01-0124-FEDER-020377, with the FCT reference PTDC/EEA-CRO/119000/2010.


Coordinated Control of Stimulus-Driven and Goal-Directed Multisensory Attention Within the Context of Social Interaction with Robots



When interacting in socially-relevant applications, robots are expected to engage with humans while displaying attentional behaviours that resemble those of their interlocutors; as a matter of fact, they are supposed to be able to assess intentionality and to be, themselves, intentional agents. Several solutions have been proposed for providing social robots with the ability of engaging in joint attention, the ability to share attention with another agent towards the same object or event, one of the most primal of social interactions. However, they have yet to appropriately capture some of the most crucial skills involved, such as the multisensory nature of active perception and attention, its inherent uncertainty, or the processes responsible for the emergence of an intentional stance. C onsequently, social robots have only been able to instil a sense of intentionality and reciprocity for very specific and constrained social scenarios. We therefore propose to research an integrated probabilistic framework to deal with the endogenous and exogenous coordinated control of stimulus-driven and goal-directed multisensory attention within the context of social interaction.

FCT Contract PTDC/EEI-AUT/3010/2012

Funded by COMPETE – Programa Operacional Factores de Competitividade e do Quadro de Referência Estratégica Nacional – QREN

Budget: 72.488,00€

Principal Investigator: João Filipe Ferreira


TICE Healthy


Description: The project “TICE.Healthy, it fits in” Generations “of the regulation of the System of Incentives for Research and Technological Development (RTD), and is one of theanchor projects inserted in the pole TICE.PT – Pole of Competitiveness and Technology.

Aims to provide health services in the market focused on the user, using for that theinternet infrastructure and thereby converging to the availability of such solutions on the market. To do this, planned the development of a platform of products andservices for the Health and Quality of Life, serving as a catalyst and facilitator inco-sponsoring companies and other external to the project, and to create services that use this as support Internet infrastructure policy.





Leader: Critical Health
Participants: AIBILI, HUC, FCTUC, Neuroeye – Electromedicina e Psicofisiologia da Visão, Lda


Optical Models for the Human Retina


Portuguese title: Modelação óptica da Retina Humana na Saúde e na Doença: da estrutura à função

Leader: AIBILI
Participants: ISR-UC, Dept. Mathematics-UC, Dept. Phisics – UC

Recently approved by FCT


TICE Mobilidade


Description: The project “TICE.Mobilidade, fits in” Generations “of the regulation of the Systemof Incentives for Research and Technological Development (RTD), and is one of theanchor projects inserted in the pole TICE.PT – Pole of Competitiveness and Technology.

It was proposed taking into account that the area of ​​mobility has been identified as apriority in the European economic and social policies (the European economy losesannually about 100 billion euros, or 1% of EU GDP, due to problems mobility) andthe action plan of TICE.PT have elected the “Mobility”as one of their key areas for action.

Aims to provide marketing services focused on user mobility, using a digital platformfor such a facilitator and thereby converging to the availability of such solutions on the market.

Achieving this objective involves the exploration of new solutions to urban transport more efficient and comprehensive, through the use of information andcommunication technologies (ICT) to integrate the various solutions available, anenvironmentally friendly way, and with more energy efficient quality for users, in coordination and cooperation with other national initiatives.


EuroAGE – Innovative Initiatives for the Promotion of Active Ageing in the EuroACE Region



The EuroAGE project aims at fostering autonomous healthy life, mainly in elderly people but also in people with some degree of dependency, through the promotion of innovative initiatives based on information and communication technologies provided by the patterns participating in the project. The project involves Portugal-Spain cross-border cooperation of partners from EuroACE region, a region that covers Centro and Alentejo regions in Portugal, and Extremadura region in Spain, which is a territory in Iberian Peninsula heavily affected by the ageing of the population. The research team from ISR – University of Coimbra that participates in the EuroAGE project will demonstrate and validate with elderly people the use of social robots and smart homes as key technologies to promote active and healthy ageing, through cognitive, socio-emotional stimulation and physical activity promotion, thus contributing to quality of life enhancement and increased healthy life expectancy, mainly in the elderly population.


  • ISR – University of Coimbra (Portugal)
  • Universidad de Extremadura (Spain)
  • Centro de Cirugía de Mínima Invasión Jesús Usón (Spain, Coordinator)
  • Instituto Politécnico de Castelo Branco (Portugal)
  • Instituto Politécnico da Guarda (Portugal)
  • Cluster Sociosanitario de Extremadura (Spain)

Principal Investigator: Rui P. Rocha. 


STOP – Cooperative Robotic Securitie



Over the past years, the private security sector in Europe has been growing, representing an annual turnover of about 35 billion euros. This growth makes this sector as a market with potential to be exploited in an innovative technological perspective. On the other hand, security tasks are monotonous, repetitive, have low value-added for humans, and can be potentially dangerous in some scenarios, thus presenting a high potential for human agents to be replaced by robots.

The STOP project presents the implementation of a new technology in the development of novel surveillance systems, with prospects of technical and economic feasibility and conditions of replicability, by exploiting the knowledge associated with new robotic technologies. The project, called Cooperative Robotic Securities ("Seguranças robóTicos coOPerativos" in Portuguese, or STOP), follows the research promoted at the University of Coimbra by the Institute of Systems and Robotics (ISR-UC), in order to demonstrate, through the start-up company Ingeniarius, the economic and technical advantages associated with cooperative multi-robot patrolling. Such evidences will be put in place by the Technological Center of Ceramics and Glass (CTCV), before a specialized audience (e.g., security teams), and in real situations (e.g., surveillance of buildings), validating the scientific methodologies developed in a technological perspective for commercial value.

The STOP project aims at implementing several scientific contributions within distributed multi-robot patrolling and the development of innovative technical features in order to adapt the robotic system to real scenarios, namely the automatic perception of abnormal situations, as well as the resilient operation during long periods of time.


APSUBA – Active Perception for Scene Understanding and Behaviour Analysis (APSUBA): An application for Social Robotic

Timespan: 01/2011-12/2012



Active Perception for Scene Understanding and Behaviour Analysis (APSUBA): An application for Social Robotic. Wiki.

The aim of the presented project is to develop an active perception system of the environment, acting as an agent inside a heterogeneous sensor network, for scene understanding and behaviour analysis. The proposed perception system is composed of two different mechanisms: an active visual perception system and a metric perception system acting inside a network of heterogeneous sensor such as 3D laser scanner, Inertial Measurement Unit (IMU) and camera. Both two systems will be calibrated. The sensorial fusion allows detecting the regions of interest using active vision (e.g. changes in the scene, human-robot interaction, human/robot motion, etc). Metric information will be used for later segmentation and 3D modelled stages. Moreover, data fusion will be applied for both heterogeneous sensor calibration and perception. Finally, the models will be used for scene recognition and behaviour understanding. In order to obtain the relevant elements in the scene, a perception-based grouping process will be employed, which is performed by a hierarchical irregular pyramid. Using the information given by the visual mechanism, the metric perception system will provide 3D information of the interest sector, through developing a multi-layer homography-based reconstruction approach. The segmentation in large datasets will be achieved using clusters provided by Gaussian Mixture Models (GMM). These segments will be modelled using high level geometric features (superquadric surfaces), which will be used for the last stage of the system: scene understanding and behaviour analysis using Bayesian rules.

The proposed project will provide contributions in different topics like mobile-structure sensor network, heterogeneous sensor calibration, localization, scene recognition, 3D reconstruction, active perception, sensorial fusion or human behaviour understanding. The results of this project are also interesting in other research fields (e.g. smart environment).


VISOR – Visual Perception System for a Social Robot

Timespan: 09/2005-09/2006


Description: The aim of the research study of the VIsual System for a SOcial Robot Project (VISOR)
is to generate in-depth knowledge of the visual human oriented perception system for a social robot. Our research study will investigate how the requirements for accomplishing a visual task determine the optimal architecture of a vision system. The issues that will be considered are: human and object representation, selection of low-level and high-level vision modules and control schemata.


RAC – Robótica Académica de Coimbra

Timespan: 10/2003-03/2005


Description: Robot construction for student teams participation on the Small Size League of the robotic football competiiton
Realização de robots para equipas de estudantes para participar em competições de futebol robótico na Small Size League da competição RoboCup.


DIVA – Dirigível Instrumentado para Vigilância Aérea

Timespan: 01/2004-01/2006


Description: DIVA is the portuguese acronym for “Dirigível Instrumentado para Vigilância Aérea”.

This is a project in the area of aerial robotics, an area which has justified a growing interest in recent times, with very different application objectives, namely in the civilian domain, for monitoring of roads, detection of forest fires, inspection of power lines. Among the advantages of the airship solution, we may cite its natural stability, low operational costs, reduced pollution, and the ability to move at very low airspeeds or even hover.

The project involves a team of researchers from IDMEC/IST, from ISR/Coimbra from the University of Minho.


Mapas Multimodais para Robótica Móvel

Timespan: 10/1999-9/2001

Description: Multisensor fusion algorithms for moving sensors. These sensors are mounted on mobile platforms in the interior of buildings.


SIVA- System for Active Surveillance

Description: Neste projecto pretende-se desenvolver sistemas de vigilância para interiores de edifícios e baseados em plataformas móveis dotados de capacidade de visualização e cooperação.


CROM- Célula Robótica de Montagem

Description: Multisensor distributed processing structure(including vision) to control a flexible assembly cell.


VARMA- Visão Activa para Robótica Móvel Autónoma

Description: Sistema de visão com possibilidades de controlo da posição dos sensores de visão no espaço e baseado nos princípios da visão activa.


FSAR- Fusão Sensorial Para Aplicações de Robótica

Description: Multisensor fusion techniques for robotic applications.


ESPRIT Exploratory Action – 5673

Description: Sistemas Multi-Sensoriais. No qual foram estudados aspectos de integração de informação multi-sensorial.


ESPRIT Basic Research Action – 3278

Description: Visão Tridimensional para Aplicações em Robótica e no qual foram estudados técnicas de integração de informação tridimensional em robótica.


EURON – European Robotics Research Network

Timespan: 12/2000-12/2001


Description: EURON stands for the “EUropean RObotics research Network”.

This project’s purpose is to implement and maintain a European network of excelence in the area of Robotics, with the objective of coordinating research, education and collaboration between the academic and the industrial world within Europe.

The main goal of the project is to promote the development of Robotics, so that Europe becomes a worldwide player in this domain.



Timespan: 1996-1999


Description: Durante a segunda fase do projecto PO-ROBOT teve a responsabilidade de coordenar a equipa do pólo de Coimbra do Instituto de Sistemas e Robótica afecta ao projecto NATO Science for Stability – PO-ROBOT. O projecto envolvia equipas dos pólos de Lisboa (I.S.T.-U.N.L.) e Porto (F.E.U.P.) do Instituto de Sistemas e Robótica. Projecto de robótica móvel financiado pela NATO (programa Science for Stability) em 1994 e designado POROBOT no qual participaram também empresas portuguesas. Na primeira fase deste projecto foram desenvolvidas as técnicas necessárias para a fabricação, em Portugal, de um robot móvel. Na segunda fase foram desenvolvidas técnicas para realização de um robô móvel com algum grau de autonomia (planeamento automático de trajectória e capacidade de evitar obstáculos) para operar em ambiente industrial.


Traffic Surveillance: Sistema Avançado de Vigilância de Tráfego Rodoviário

Timespan: 1999-2002


Description: The authors propose to build an automatic traffic surveillance system which will learn by observation the rules drivers use to control their vehicles. The system will be able to assess a driver’s performance relative to that of his/her peers and to diagnose the reasons for any discrepancy.
The system will consist of a number of stationary surveillance cameras connected to ordinary personal computers. The computers will track any vehicles in and feed the results of the tracking into an established machine learning program. By assuming that any acceleration of a car (i.e. speeding up, slowing down or turning) indicates a control input to the car the learning program will be able to associate the driver’s actions with the car’s current situation (i.e, the state of any nearby vehicles, etc.).

The proposed project will produce a car tracker with capabilities significantly in advance of those presently available. Cars will be tracked with multiple cameras and over large areas with a tracker able to detect the cars’ signal lights The emphasis has been on increasing the reliability, robustness and range of applicability of the tracker. Real-time performance of the trackers would be a welcome capability, and one to aim at eventually, but the system goals can all be met by slower off-line processing.

The results of the tracking will be compared to independently measured data in order to validate the tracker. An inertial sensor for robot navigation has been developed and will be available to this project, and an agreement in principle from the Coimbra traffic police to use their doppler-radar equipped surveillance cameras as been achieved.

The proposed project will deliver a software system able to track cars over large areas using multiple communicating trackers and learn the rules used by the drivers of those vehicles. The cars turn and brake signals will be recovered and tracking will be possible in a greater range of weather conditions than possible at present. It is anticipated that the models learnt by the system will be usable to assess and diagnose the performance of either individual or groups of drivers.


SIRMA – Sistema Inercial para Robótica Móvel Autónoma

Timespan: 1995-1998

Description: . Dentro dos modernos métodos de produção encontram-se os processos flexíveis de produção que normalmente englobam a utilização de manipuladores, AGV’s e robôs móveis. Actualmente grande parte dos processos utilizando robôs móveis e AGV’s baseiam-se em sistemas que executam sequências previamente programadas com um grau limitado de interacção com o ambiente. Este tipo de aproximação apresenta o problema da programação dos dispositivos de modo a respeitarem a geometria do local. Nas aplicações em que não existe uma geometria do local já conhecida é necessário que o robô disponha de capacidade de localização e percepção para uma correcta navegação. Este tipo de aplicações, para serem executadas de forma autónoma, implicam a utilização de sensores para uma correcta execução dos movimentos. Utilizando a informação fornecida por sensores inerciais, tais como acelerometros ou giroscópios, é possível construir um sistema de navegação inercial baseado em medidas directas feitas com os sensores e no interior do veículo. Estes sistemas apresentam propriedades interessantes pois as medidas realizadas são independentes da informação sobre o local onde se situa o veículo, da forma de contacto das rodas com o solo e não utilizam medidas sobre a energia radiante do local ou emitem energia para realizar medidas. Um sistema deste tipo pode ser facilmente utilizado noutras aplicações que envolvam medidas de velocidade ou de aceleração de um corpo em movimento.


Digitalização tridimensional da imagem da coroide humana

Timespan: 1992-1996

Description: O projecto tem como objectivo o estudo da circulação e estrutura coroideias, no homem e no animal, através da digitalização bidimensional de imagens de angiografia com verde indocianina, obtidas com o vídeo-angiografo. Este estudo irá permitir a identificação da coriocapilar e dos grossos vasos, durante a fase de trânsito coróide, caracterizar melhor topograficamente a circulação coroideia e detectar lesões estruturais nas diferentes camadas.


TELE 3D – Um Scanner 3D para tele-colaboração

Timespan: 07/2001-07/2002


Description: Three-dimensional surface reconstruction from twodimensional images is a process with great potential for use on different fields of research, commerce and industrial production.

This project comprises the study and development of systems which implement the aforementioned process, with potential application in tele-colaboration and tele-manipulation applications.



Timespan: 12/2001-12/2004


Description: Implementation of novel transportation technologies for electricity-driven vehicules and promotion of their use in urban environments.


Multi-Modality Medical Image Registration for Eye Diagnosis

Timespan: 02/2004-06/2006

Description: Diabetic retinopathy is the leading cause of legal blindness in developed countries. Ten percent of the EU population are expected to develop diabetes by the year 2010 while in the USA there are, currently 100.000 new cases each year. With the incidence of diabetic retinopathy among the diabetic population, a large socio-economic impact is expected.

Each method of diagnosis focuses on a particular feature of the retina. This gives a partial view of the changes occurring in the retina in a given time whereas an entire overview is essential for a better diagnosis and to understand the changes that are occurring and are expected to occur in the near future.

In order to achieve a broader overview of the changes occurring in the retina, it is necessary to correlate several complementary examinations: fluorescein angiography, confocal scanning laser angiography, retinal blood flow, retinal thickness and retinal leakage.

Registration of the modalities involved has to be automatic so that it can be used in the daily practice in the clinical environment. Differences in scale, resolution, distortions, rotation and translations must be taken in account.

Registration methods are classified as feature or intensity based.

The former is based on the segmentation of homologous geometric landmarks present in the images while intensity based registration searches a space transformation for the maximum of a given measure based on a particular criteria of evaluation of similarity. This similarity is classified as: identity relationship; affine relationship; functional relationship; and statistical relationship.

A physical model will be established so that the knowledge on structures being measured can be incorporated in the process. Whereas the image corresponds to light coming from a layer of given depth (different at each point), for retinography the information gathered from the same retina with retinal flowmeter takes into account a light from different layers due to the confocality of the system. This is to say that different entities are being measured with different methodologies.

A physical model will be established to provide information on what to expect from each modality, its relation with the whole reality, and what to extent the information is related and in which aspects.

The approach to be taken in this project is to test several registration techniques for each modality. The techniques to be tested are the ones that most likely shown any relation to the physical model previously developed.

In order to address all the above-mentioned problem the physical model will be established incorporating the knowledge of the physical eye and the acquisition technology, as well as its statistical properties.


Structure-function imaging for quantitative phenotyping of retinal degenerations

Description: Following previous work on the field of medical imaging on the topic of ophthalmology, new opportunities and needs for data integration have emerged. The topic is now broader and follows the global direction of modern medicine.

This multidisciplinary project aims to understand how patterns of degeneration in the adult human retina are related to the organization of its functional mosaics. It is important to establish new quantitative strategies that can explore structure-function relations in the retina, which may help clarify why retinal degenerations exhibit considerable inter- and intra-familial variation in their expression. The goal is to better understand the normal function of different retinal pathways and apply this knowledge to the improvement of quantitative phenotyping.

This is only possible by taking into consideration the possibility of data integration, i.e., by merging information from the structure and function, and therefore a new toolbox will have to be provided to the medical community.

Technology transfer actions at Systems and Automation Lab (IPNLas ) from the Instituto Pedro Nunes ( IPN ) – University of Coimbra ( ).

Concluded Projects

Technology transfer actions at Systems and Automation Lab (IPNLas ) from the Instituto Pedro Nunes ( IPN ) – University of Coimbra ( ).