How Do You Measure the Level of Automation in Decision-Making

Generic Level for All Tasks

Table 2 shows a highly practical classification for measuring the degree of automation [2]. This method is based on four generic tasks, with each level assigning a task or set of tasks to either the human or the computer, or both. In this classification, automated decision-making begins at level 8.

Task-Specific Levels

Assigning systems to a generic level (Table 2) is frequently challenging in practice. The Level Of Automation Taxonomy (LOAT) in Table 3 attempts to address this by defining levels per task [3].

The authors in question separate four tasks: gathering information, analysing information, selecting actions, and carrying out actions. For each task, 5 to 8 custom automation levels have been developed. These begin with a default level (0) that corresponds to manual task execution and progress to full automatic job. Level (1) presumes human input with simple external support, which is not truly automatic. At level (2), there is real automation.

LOAT demonstrates that there is no such thing as a 'generic' level of automation for a system, but that it should always be evaluated on a task-by-task basis. An automated system can also support some of these four tasks, each with a varying level of automation. Finally, this system demonstrates how automation affects how humans are supported in accomplishing their tasks.

Criticism of LOAT

Aside from the disadvantage that it is difficult to classify an algorithm unambiguously using the LOAT approach, two perspectives are also absent.

For example, the function of people and computers in building an algorithm for an automated system is still absent. The person in an expert system defines the rules and steps that an algorithm employs. This can also be greatly automated by employing Artificial Intelligence (AI).

This construction of a decision-making model could be added to the classification in Table 2. This extension has two levels: modelling by humans (expert systems, level 0) or computers (AI, level 1). LOAT also disregards ethical considerations. This also makes it simple to broaden the classification. These ethical aspects have three levels: humans have an overview, human supervision, and full automation, in which human intervention play no role.

Symbio6 & Levels of Automation

The level of automation is a key criterion for describing and comparing automated decision-making algorithms. Symbio6 assists clients with choosing the most appropriate classification for their situation.