It’s a hot August day at the Maryland State Police Training Academy. The sun is shining bright and there’s a constant buzzing in the air. It’s not insects—though those are certainly out and about, as well—it’s small unmanned aircraft systems (sUAS), also known as drones.
Dozens of officials from across the country, spanning a variety of different federal, state, and local law enforcement agencies, have gathered together for intensive training. The “Advanced Open/Obstructed Test Proctor Course for Evaluating Drone Capabilities and Remote Pilot Proficiency” was developed by the National Institute of Standards and Technology (NIST) in conjunction with the Science and Technology Directorate (S&T). The goal is right there in the title: evaluating capabilities and proficiency. Competent drone piloting is critical when lives are on the line; these devices are used in numerous law enforcement operations including search and rescue and counter IED (improvised explosive device) efforts.
“We first developed these test methods with the idea of helping the government to identify and test ground robots to make sure we have a standardized method and we’re buying the best,” explained S&T Standards Manager Kai-Dee Chu, PhD. “We have been using them to test drones for procurement purposes, and the first responders found out for themselves that these standardized test methods are even better than their training courses. So, they adopted these test methods and it’s just caught on like wildfire—not just in the United States, but now in Canada, in Korea, in Japan … the test methods are used by all these first responders.”
So far, this training has been offered three times—in California, Texas, and Maryland—since it was first introduced in January 2022, and it has led to more than 400 certified proctors. The next course offering will be in New Jersey this November. It consists of 24 hours of classroom and hands-on flight instruction over three days to “train the trainer,” so newly certified proctors can take what they’ve learned back to their home agencies and subsequently certify their drone operators. It’s a wonderful domino effect of enhanced officer expertise and increased public safety. The course follows NIST test methods that have been adopted, or are under consideration for adoption, by ASTM International, National Fire Protection Association, the Airborne Public Safety Accreditation Commission, the Civil Air Patrol, and many other federal, state, and local public safety organizations.
“When we started, there was no measurement science or standards infrastructure available to objectively evaluate drone capabilities or remote pilot proficiency, so we filled that void,” said Adam Jacoff, NIST project leader for Emergency Response Robots and chair of the ASTM E54.09 Subcommittee on Response Robots. “After helping to guide purchases, these standard tests then support credentialing of remote pilots. Although these drone test methods are specifically designed to help emergency responders and public safety organizations maintain a safe operational standoff while performing extremely hazardous tasks, they similarly support a wide variety of commercial and industrial applications. All pilots flying in the national air space need to demonstrate they can maintain positive aircraft control while performing operational tasks in complex and often hazardous environments.”
The course includes both open and obstructed test lanes, meaning with and without line-of-sight, as well as realistic operational scenarios—all using established assessment standards for scoring. Flights are conducted both during the day and at night so pilots can be ready for anything. As vigorous as it is, the tests are inexpensive (using equipment like plastic buckets available at any hardware store), easy to conduct, and require less than 30 minutes to complete. Attendees learned how to fabricate the test apparatuses, conduct trials, and embed the same scoring tasks into their own training scenarios. Upon completion, they know everything they need to replicate the course back at their home agencies.
One of the realistic operational scenarios that participants played out in Maryland was based on an actual emergency call from earlier this summer involving an abducted child and an abandoned vehicle in a rural area.
As Corporal James Lantz with the Maryland State Police put it, “The scenario that we're doing out there, the open area search, is basically the exact same scenario of a mission that we flew about two months ago. We had an Amber Alert. The vehicle was found in a field…surrounded by trees and everything like that. So, we can sit there and say… ‘This is exactly what we faced. These were the hazards that we had. These are the issues that we had.’ We replicated that here, in a controlled environment, a testable environment that we can use to measure that these guys are able to perform.”
Drone operators were instructed to thoroughly inspect the vehicle as if searching for any signs of people, weapons, or even explosives within. Participants were required to take photos of the various targets tapped to the car with their drone’s camera.
To become certified as a proctor, participants must pass a written quiz with a grade of at least 80% and submit scores from trials they proctored for other remote pilots. The officers work in teams of three, rotating through each role of pilot, proctor, and visual observer. The pilot maintains control of the drone, calling out each intention of movement before doing so and calling out each bucket alignment as they make their way through the obstacles. The proctor scores the pilot. The visual observer maintains sight of the aircraft and surroundings, repeat’s the pilot’s intention of movement to confirm, and calls out corrections and warnings as necessary.
In addition to testing the skills of the drone pilots, the course is a great way to see what the drones can do. As David Battaly from the Mississippi Emergency Management Agency said, “It's a practical measure of not only the pilot's proficiency and ability to operate his aircraft, but the capabilities of that individual aircraft. There's a myriad of drones on the market, and each drone is a little bit different. So, it measures the pilot's proficiency, how well he can manipulate his drone, and the capabilities of what his drone can actually achieve.” Seeing how different devices perform helps inform procurement decision making so law enforcement agencies can purchase the best tool for the job.
The course also helps manufacturers improve their products. Private companies are able to participate in the same training, running the same scenarios, but instead use it as an opportunity to de-bug their systems and see how they compare to other devices on the market. A manufacturer is able to run through the evaluation process, determine the best that their system can do, and identify ways to boost performance. This may mean realizing they need more robust optics or a more intuitive user interface so pilots will have an easier time using their device, score higher on the course, and be more likely to recommend it to colleagues.
Thanks to the easily repeatable nature of the course and the uniformity and consistency offered by the standards, drone operators across the country will be able to compare “apples to apples” when assessing capabilities during a response. This approach encourages collaboration and ultimately leads to a safer public.
“It's important because we want to have certified pilots to fly these drones to make sure they operate safely and proficiently,” added Chu. “But more than that is now that we have standards, we have common ground to talk about these capabilities. And these standards are not stagnant. We need to revise them every two or three years, but we can always have the latest, the best test methods for our operators and for our first responders. So that's the beauty of using these international standards for these training facilities because they are well tested, they're well accepted, and even after we are gone, these standards will still be there.
The drone standard test methods are expected to be accepted by ASTM by spring of 2023 and next up will be underwater test methods for aquatic robots.
