A Flexible and Common Control Architecture for Rolls-Royce Marine Cranes and Robotic Arms

Publications

Filippo Sanfilippo, Lars Ivar Hatledal, Houxiang Zhang and Kristin Ytterstad Pettersen. A Mapping Approach for Controlling Different Maritime Cranes and Robots Using ANN. In Proceeding of the IEEE International Conference on Mechatronics and Automation (ICMA), Tianjin, China. 2014, 594–599.

Filippo Sanfilippo, Lars Ivar Hatledal, Hans Georg Schaathun, Kristin Ytterstad Pettersen and Houxiang Zhang. A Universal Control Architecture for Maritime Cranes and Robots Using Genetic Algorithms as a Possible Mapping Approach. In Proceedings of the IEEE International Conference on Robotics and Biomimetics (ROBIO), Shenzhen, China. 2013, 322–327.

Filippo Sanfilippo, Hans Petter Hildre, Vilmar Æsøy, Houxiang Zhang and Eilif Pedersen. Flexible Modeling And Simulation Architecture For Haptic Control Of Maritime Cranes And Robotic Arms. In Proceedings of the 27th European Conference on Modelling and Simulation (ECMS), Aalesund, Norway. 2013, 235–242.

Video

Abstract

This project focus on the developing of a flexible and general control system architecture that allows for modelling, simulation and control of different models of maritime cranes and, more generally, robotic arms by using the same universal input device regardless of their differences in size, kinematic structure, degrees of freedom, body morphology, constraints and affordances.

Project partners

• Rolls-Royce

• Marine AS, Per Huse, AAUC

Duration of the project

2 years, from January 2011 to December 2013

Short project description

Maritime cranes are widely used to handle and transfer objects from large container ships to smaller lighters or to the quays of the harbours. The control of robotic cranes is always a challenging task which involves many problems such as load sway, positioning accuracy, suppression, collision avoidance, and manipulation security. Low control flexibility and non-standardisation are two crucial points of current crane control architecture. There is no standardised uniform control input interface or flexible and common control architecture to deal with all possible cranes with different kinematics – even if the cranes are from the same industrial company. As a result, experienced human operators are only good at controlling some types of cranes even after many years of training. If operators have to control other crane systems, more professional training is absolutely necessary in order to get familiar with using the new input device with the different control strategy since various cranes have different degrees of freedom (DOF) and kinematic models as well. This is a time-consuming and embarrassing situation in the maritime industry. The underlying idea in this project consists of developing a flexible system architecture that allows for the control of different robotic arms using the same haptic device regardless of their kinematic structure, degrees of freedom, body morphology, constraints, affordances. The project will improve the safety of demanding marine operations. The results from this project will give unique measurable knowledge on how to improve the human factors, safety and usability in controlling robotic arms and cranes in maritime applications. Furthermore, the results will provide the maritime industry with guidelines that consist of concrete suggestions on how to support crane operators at crucial times during the operation.

Objectives / aims for the project

At Rolls-Royce, to date, there is no successful crane control architecture featuring both sufficient manipulation flexibility and the necessary standardisation needed to achieve an optimal combination of working efficiency and security. This situation urgently demands innovation in advanced crane control methodology. The proposed project has the following three main objectives:

1. Develop a flexible and common control architecture for Rolls-Royce marine deck cranes / robots, which offers working efficiency and safety in marine operations;

2. Integrate and test the proposed control architecture with the “crane/winch simulator” developed by the Offshore Simulator Centre (OSC) of Aalesund;

3. Develop a common haptic control device for various types of Rolls-Royce marine deck cranes and robots.

The sub goals the project intends to fulfil are:

• Investigate the kinematic topology of all cranes from Rolls-Royce Marine AS;

• Establish a research set-up for the control architecture concept;

• Investigate synergies strategy in order to find a flexible kinematic mapping methodology;

• Automatically generate kinematics for calculating the positions and speeds for the relevant cranes;

• Implement the kinematics mapping with a haptic input device to make a demo system;

• Integrate the control architecture with the OSC environment;

• Develop a novel common haptic control device for Rolls-Royce marine deck cranes and robots;

• Improve safety and control by reducing ambiguities and emphasising simplicity;

• Transfer the project results to a real control system for Rolls-Royce Marine AS.

Review of results achieved compared to objectives

After two years working together with our partners, we have achieved the following results:

1. We have developed a flexible and common control architecture for Marine deck cranes / robots based on artificial intelligence algorithms, which offers working efficiency and safety in marine operations;

2. OSC and AAUC have worked together and integrated the current control architecture with the simulator developed by the Offshore Simulator Centre (OSC) of Aalesund;

3. We have designed and developed a novel haptic control device for various types of Rolls-Royce marine deck cranes and robots.

In details, we fulfilled the following achievements:

• Investigate the kinematics of all cranes;

• Establish a research set-up for the control architecture;

• Investigate synergies strategy in order to find a flexible kinematic mapping methodology;

• Automatically generate kinematics for cranes;

• Implement kinematics mapping with the haptic device;

• Implement the numerical Jacobian (Extra);

• Test the effectiveness of the control algorithm (Extra);

• Improve safety and control;

• Try all control functions with force feedback (Extra);

• Develop a new haptic control device for Rolls-Royce marine deck cranes and robots;

• Study the OSC protocol to communicate;

• Implement a client to the kukavarproxy in Java (Extra);

• Investigate the architecture system of the OSC;

• Complete project summary and transfer the result to the real robotic system.

From the items listed above, we have finished all aims and sub-goals we proposed. According to the real project research situations, we add several extra sub working tasks in order to fulfil the project as good as possible. The task related to design a novel haptic device is finished already. The manufacturing of this haptic device is delayed by the cooperated partner. It will be delivered to us in May 2014.

Description of most important research and development activities and which partners are the central in the process

Up to date, sending simple instructions to give information such as spatial coordinates or even joint angles is still a common way to control robotic arms and big cranes in maritime applications. Since this sequence of commands is not easy for humans to understand, it is hard for the operator to give correct commands to the crane. Furthermore, during an operation, sometimes a human operator has to interact with many different crane systems that have different interface styles. This demands a great deal of adaptability from the operator since not all of them are designed to be proportional to the mental and physical capabilities of the operator. This leads to a problem of dealing with too much information which increases cognitive loads. Poor ergonomic design of control devices can also result in physical strain. To deal with these problems to improve safety of demanding marine operations, it could be useful to employ some kind of common controller that would be much more intuitive for humans. Therefore the research and development challenges includes the followings:

• Controlling strong slave robots using innovative control laws;

• Controlling different cranes/arms using the common haptic device;

• A selection of tools based on knowledge from previous research and related literature to identify and analyse some classic approaches in mapping different embodiments;

• Approaches to reduce the stress-level (including cognitive load) of the operator in both normal operations and safety critical situations in terms of ergonomics and design-guidelines for maritime software applications.

In the project, Aalesund University College is the main contributor. This project relies on the close collaboration of the project partners. Each member should have enough authority over local resources to resolve all potential intra or inter site-related project problems.

Short evaluation of project process and resources used in the project

This project combines expertise from complementary disciplines: robotics, artificial intelligence, computer science, and the maritime industry. The interdisciplinary nature is expected to lead to fundamentally new control systems based on kinematically competent, multifunctional systems. The project is led administratively by Per Huse at Rolls-Royce Marine Deck Machineries. Rolls-Royce will set up the specifications needed for the research project and support the AAUC in the testing/verification of the results from a user’s point of view. Aalesund University College (AAUC) will be represented by their resource persons in research, teaching, supervision and training. Professor Hans Petter Hildre will be the primary contact and coordinator on behalf of Aalesund University College. The research is mainly based on research background and resource at Mechatronics lab led by Prof. Houxiang Zhang at Aalesund University College. In these areas, the research group has published over 100 journal and conference papers and book chapters. Huse Engineering provides detailed set-up of existing control device functionalities. They will also provide needed information about existing crane topologies and data. In addition, there are close collaboration with the Offshore Simulator Centre (OSC) in order to integrate the proposed system with the current simulator for researching and training. The research will pave the way towards new applications in the maritime domain, particularly towards crane handling and manipulation.This project is expected to contribute to improving the competitiveness of Norwegian scientific research not only in the area of robotic crane manipulation but also in its application in the maritime domain.

Description of value of results (research, education, industry etc.)

The proposed project will address the development of a flexible control architecture and a common device for all different marine deck cranes/robots in marine operations. The project will therefore be unique and it will strengthen the maritime cluster in the region and nationally. From a research perspective, as far as we are aware, to date there are no similar crane control approaches available in Norway, nor in Europe or in the US to provide a flexible and common control architecture for different diffident marine deck cranes and robotic arms that have different kinematics models and DOFs. The system will offer working efficiency, manipulation flexibility and safety in marine operations. The project will therefore be unique and it will strengthen the maritime cluster in the region and nationally. From an industrial application perspective, developing a common control device for Rolls-Royce marine deck cranes and robots in real applications is a very promising innovation for maritime industry. The system proposed in the project will relieve human operators of their work-related stress and make efficient and automatic crane controlling possible and easy. Additionally, it will improve the technological level and productivity of the maritime industry. From an educational perspective, we will integrate the flexible crane control architecture with the training simulation system in the OSC so that the students at AAUC will have a good opportunity to get professional training and preparation for work in the future.

Description of which results are expected after the project ends

Development of high performance cranes for safe deck operations and advanced marine operations is the strategic core at Rolls-Royce Deck Machinery. The turnover for the deck machinery unit is more than 2 billion NOK and the crane segment is expected to grow. Advanced control and usability allowing for efficient and safe operations is the key differentiator. Project success is therefore important. The proposed project will highlight the need and the value of bringing robotic technology and interaction design into the maritime sector. A flexible and common control architecture for marine deck cranes will offer working efficiency and manipulation safety in operations. The human operator will feel comfortable and at ease for crane manipulation. Lowering the operator’s stress and use-related difficulties will provide a safer and more efficient operation. This will reduce the human error rate related to incidents, which can reduce the likelihood of the operator making the wrong decision leading to accidents that may cause damage to the crew, the vessel, and offshore installations also increasing the risk of environmental and social damage. The project will add a new dimension to the company’s profile and offer further means to development for the company beyond this initial investment. Technology developed in this project will be integrated into the simulator at OSC. It will then be implemented step-by-step on real cranes. Several types of cranes of a variety of sizes are in production. A novel “parallel bar” crane is in the development stage and it is assumed that it will be an important supplement in the Rolls-Royce crane assortment. The future step is to implement the new technology on the parallel bar crane first and then implement it on the other cranes. Implementation will be done as fast as possible and is expected to be tested on-board at the end of this project.

Official project page

For more information about this project visit its official page.