How to Plan a Successful Autonomous Systems Demonstration

Australian Droid & Robot demonstrating at EPE MILTECS Facility at CSIRO and Queensland Defence Science Alliance ‘Human Teaming and Response Robotics Standardised Testing, Evaluation and Certification Interactive Showcase’ Pullenvale 21 April 2022

Australian Droid & Robot demonstrating at EPE MILTECS Facility at CSIRO and Queensland Defence Science Alliance ‘Human Teaming and Response Robotics Standardised Testing, Evaluation and Certification Interactive Showcase’ Pullenvale 21 April 2022

Demonstrating the viability of artificial intelligence (AI) requires thoughtful construction and communication of both social and technical aspects.

Demonstrations are both scientific experiments and social events designed around achieving buy-in for the technology. When it comes to AI, demonstrators face the challenge of conveying the smarts of the system and the role of human intent.

Two challenges of demonstrating the potential of AI are:

  • Performing decision-making: working out how best to show cognitive and social decision making through complex autonomy demonstrations in a way that makes intelligent performance and errors understandable
  • Showcasing ability to abide by human values and intent: working out how best to design a demonstration that shows the ability of a system to abide by decision making norms, including commander’s intent, military objectives, and ethical, legal, and safety-focused frameworks

Demonstrations are performances that include social and technical elements, such as: actors (humans, UI, software, networked systems and machines); enabling technologies (mechanics, comms, screens, instructions, consoles, connectivity, controls, batteries/fuel, security, lights, tents, generators); rituals (cultural behaviours, safety processes, signals); a choreographed narrative (CONOPS, narrative flow including objectives, cause and effect, conflict and emotions); planned, opportunistic and incidental interactions (stories, networking); evaluative criteria (expectations, key performance indicators); and goals (training/education, buy-in, positive emotions, media and communication outcomes, future investments, lessons learnt).

The TAS Ethics Uplift Program is conducting a research project that will develop and test an Autonomous Systems Demonstration Canvas to help optimise human understanding and buy-in for technology developers and investors. The Canvas will help tackle the opacity problem facing AI, for instance the way AI can obfuscate the rules, reasoning, and justifications underlying human-machine decision making as it occurs.

The first iteration of the ‘Autonomous Systems Demonstration Canvas’ V.1.1 (ASDC) was launched at the Queensland Defence Science Alliance (QDSA) Human Teaming and Response Robotics Standardised Testing, Evaluation and Certification Interactive Showcase at CSIRO’s Queensland Centre for Advanced Technologies’ MILTECS facilities on 21 April 2022.

The Canvas provides a scaffolding framework to help technology developers efficiently prepare, practise, and perform impactful technology demonstrations to diverse stakeholders to achieve diverse goals including attracting investment, showing technical progress, and getting publicity.

The intention is for the Canvas to inform future trials as a tool to support impactful demonstration planning for TAS programs. For more information on the Autonomous Systems Demonstration Canvas contact Dr Kate Devitt kate.devitt@tasdcrc.com.au.