1. Introduction
The situation awareness module is able to detect critical events based on contextual data coming from the agents including the robot, the process, the task and the environment. This module uses ontologies to organize the perceived data and relies on a reasoning system and a planner in order to generate the relevant actions that could be a new action plan for the appropriate agents or a warning for the operator.
  1. System delineation

a. Context

The ontology-based approach for ensuring situation awareness represents the knowledge needed in the scope of the task and/or the process including the agents (humans and robots), the sensors, and the environment. This knowledge is then checked by a rule engine to check its consistency. Then, a planning phase is performed automatically to generate an action plan for the relevant agents.

b. Scope

Situation Awareness functionality is ensured through different levels of the HORSE framework: local and global. In the local level, when a critical event is raised by the local signal deviation module, the event is received and handled by the Local Safety Guard which generally adapts the robot behavior participating to the current task. However, if the event cannot be handled locally (e.g: inability to slow down the robot), this means that the decision was based on insufficient knowledge and that more general contextual information have to be taken into account. A critical event is then propagated to the global level which understands the situation and takes the appropriate decisions accordingly (Global Safety Guard module).

  1. Requirements and functions


The Activity Functions related to our system are the following:

AF-01 Situation Awareness that is provided by a set of sensors, cameras, and software technologies embedded in the robot and the work floor and which will be orchestrated to support the actions and tasks of the robot. These provide robust and stable control when robots are in the contact with the environment. This function describes the way robotics and humans perceive environmental elements so that there is no lack of communication between different agents (human and no humans) in the work cell and a better anticipation of matters regarding the safety of the operator. Alerts to the operator of troubleshooting or possible dangers are included, as well as confrontment of fake alerts or wrong behavior for the machines. This function enables the local prevention or solution of safety issues that may occur in a work cell due to human and robotic actors getting too close to each other. Sub functions identified for this main function are: work cell situation awareness (SF-01) and visual detection of item, intrusion, or obstacle (SF-02). SF-01 is handled by the local safety guard and the global safety guard modules of HORSE framework. SF-02 is handled by the signal deviation modules and is related to data interpretation in order to trigger critical events. Our method is based on thresholds: for each sensor for example, we know when an undesired event may happen. A set of rules are define conditions for combining different values provided by different sensors and agents.

  1. System description

The situation awareness system is integrated in both local and global awareness of HORSE framework. Global Awareness is asynchronous. It relies on events and data from the scene and contextualizes those data to take the appropriate decisions. Local Awareness is more focused on the task and its scope is limited to the robot workcell. The Deviation Monitor module of HORSE framework is the local signal deviation system. It observes deviations in a given entity: sensor, agent (human or automated), product, etc. Deviation monitor raises critical events when an undesired event may occur. This event is sent to the Local Safety Guard module, responsible for taking immediate decisions to handle the event.

The Event Processing module is the global signal deviation system. It is responsible for analyzing more general information than a specific entity: the agents, the sensors, the process, etc. This module is part of the global execution layer of HORSE framework. In case of deviations that were not intercepted by the deviation monitor and that need a global interpretation of the context, the event processing raises a critical event that is then handled by the Global Safety Guard component


  1. Demonstration scenarios

The following scenario describes a use case which involves different robots and operators sharing together the same workspace. In order to ensure the operator’s safety and avoid emergency stops, we use a situation awareness component which relies on contextual data and adapt the robot behavior accordingly.

The following figure shows a scenario where the mobile base is transporting parts from one workcell to another.  Two operators are doing different tasks. One operator is working with a manipulator and another operator is doing a manual task. The situation awareness component gathers all the data in the environment including the information about the operators and the robots positions and is able to adapt the robot behavior to avoid a collision. Figure B shows that the mobile base was informed that a possible collision may occur because the operator is leaving his workcell to go to another one. The mobile base waits then in a safe position (figure C) until the operator is far enough. The mobile base continues then its way to the target position.



  • The mobile base component sends continuously its pose to the situation awareness component. Pose is a tuple (x, y, theta) with theta being the orientation of the robot.
  • The Kinect camera component is able to detect the human position in the environment. It continuously sends the operator Cartesian position to the situation awareness component.
  • The situation awareness component analyses the received data. If a critical event is detected, the situation awareness reasoning system generates an action plan. The action plan is then sent to the relevant agents (In this example, the action plan is sent to the mobile base {stop, wait for signal to restart, restart}).

The figure below shows the scenario implementation with HORSE components.



Event Processing for ensuring operator safety: 1) Data are gathered from the devices and the agents participating to the task; 2) A critical event is raised whether an anomaly may occur; 3) Relevant information from the environment is collected by the Local Safety Guard where a reasoning about the environment is done; 4) An action plan is generated to the concerned agents. 5) Once the situation is handled, an event is sent back to the deviation monitor component. If the same critical event is raised several times or the mobile base is not able to execute the action plan, the critical event is propagated to the global level which raises an alert to the operator.

  1. Download information

Access rights shall be granted after written request (with a separate license agreement) on a non-exclusive basis, without the right to sublicense, and in case of software, in object code. The request for access right may be made during the project or up to twelve months after the end of the project. The request will specify the intended purpose (implementation of the action necessary for exploitation of its own result) and a demonstration of the rights is needed.

Access rights shall be granted against financial compensation if needed for the exploitation of a party's own results in case of direct or indirect industrial or commercial exploitation.