A team of researchers at Johns Hopkins Applied Physics Laboratory is developing a metrics-based approach to evaluating the performance of uncrewed surface vehicles and their compliance with international regulations for preventing collisions at sea.
The Performance Analysis Toolset builds on previous APL research on quantifying good seamanship for unmanned surface vehicle performance evaluation and aims to perform objective evaluations based on subjective collision avoidance protocols, the research center said Wednesday.
“To develop the tool, we sat down with subject-matter experts, asked them what qualities they look for in navigation tests, studied the COLREGS and turned that information into quantifiable metrics,” said Mike Heistand, a systems engineer and senior analyst in APL’s force projection sector.
“From there, we can feed USV navigation data into our scoring algorithm and share results with the sponsor that are consistent and objective,” Heistand explained.
Under the sponsorship of the Naval Sea Systems Command Unmanned Maritime Systems Program Office, APL is testing PAT’s validity by completing a Performance Analysis Toolset – Human Operator Comparison in collaboration with Surface Warfare School Command.
“We want to shed light on how assessments of both autonomous systems and humans may evolve to a new reality where both interact in the real world,” said Kathryn Lahman, program manager for advanced autonomy test and evaluation at APL.
