Semi-autonomous robot teams are currently being proposed for dangerous missions including weapons employment, minefield operations, reconnaissance/surveillance, force protection, ordnance detection/clearance, and urban warfare. Experimental ground robots such as the Dragon require one operator per robot while more sophisticated UAVs require multiple operators due to the concurrent demands of control, monitoring and decision-making. Kansas State University, together with Vanderbilt University, have been working on key technologies to overcoming this problem. We currently have two funded projects in this area.

Single Platform Multi-Sensor (Robot) Control

The main objective of this project is to develop the technology to allow a small number (one or more) of operators to a control multiple teams of robots in a variety of applications and to demonstrate this technology in one or more application areas.  This technology will be able to be widely applied to multiple situations where human operators need to provide oversight and control of teams of semi-autonomous robotic vehicles.

The technical basis of our approach requires integration of two technologies: organizational control and multiple robot-human interactions. Our organizational control approach is based on the organizational model of teamwork and is under development by Dr. DeLoach and Dr. Gustafson. A third member of this research team, Dr. Julie Adams at Vanderbilt University, is currently studying multiple robot-human interaction techniques. DeLoach created an organizational model that defines the elements of adaptive, goal-oriented teams. With this information, robot teams can organize at runtime to achieve its goals in the most effective and efficient manner. Adams has been involved in developing interfaces that allow a human operator to control teams of robots. 

The effectiveness of this approach will be demonstrated in a two-phase approach.  We are currently developing simulated and real-world demonstration prototypes to allow a team of cooperative robots to get into formation, move to a point following a set of waypoints, traverse danger areas, and ultimately perform area reconnaissance before returning home.  The goal of Phase I is to integrate a human into the cooperative robotic team to allow the human to control the team as a supervisor using organizational control.  This requires extending DeLoach’s organization model to allow capturing different types of user roles and capabilities and the development of techniques for providing appropriate human-robot interaction techniques. New capabilities must also be developed for the real-world robots in vision processing, multiple sensors, and interaction with humans in performance of the search, monitor and detect activities. In Phase II, the organizational control will be implemented and demonstrated using all terrain research robots in a more realistic setting with the role of humans being expanded to include that of operator and as a peer. Additionally, different approaches, such as gaming, that allow the human operator to visualize/verify the organizational changes will be investigated.

Human-Robot Teams Informed by Human Performance Metrics

Most deployed robots are currently operated by humans using teleoperation. However, future DoD concepts envision collaborative human-robot teams (HRTs) where humans and robots are deployed side-by-side as partners on missions that require tightly integrated and choreographed activities. These HRTs will require the team members to adapt to each other, the environment, and the state of the team problem solving process.

The key to HRT adaptation is providing teams with the knowledge of how team member’s performance capabilities change over time. Robot capabilities are complex, but are not as difficult to capture and quantify as those of humans. Thus, knowledge human performance factors and how they affect performance is vital to HRT adaptation. This proposal seeks to capture, model, and use human performance information to help HRTs operate at peak efficiency and effectiveness.

This research will employ existing human performance modeling tools to identify the applicability of existing HPMFs to HRTs. However, it is anticipated that existing HPMFs will need modification and new HPMFs may be required. The resulting HPMFs will be used to guide runtime HRT control software in assigning team members to mission roles. The research begins with single human-single robot teams, advances to single human-multiple robot teams, and culminates with an HRT demonstration with humans and real robots that assigns members to roles using HPMFs.

The goal of this research is to develop a theory of human-robot team organization that integrates humans and robots into a single team based on member capabilities and human performance factors.

This research will provide a common framework for incorporating humans and robots into a single team. The impact will be a clearer understanding of the applicability of HPMFs for informing the teaming of humans and robots for tightly coupled missions.