Agility Numbers Agility

Open database for agility benchmarking

As part of this publication we propose an online framework to enable researchers to share their experience with the agility-benchmark. The work leading to this benchmarking database is described and published in detail under the following link:

Benchmarking Agility for Multilegged Terrestrial Robots

We hope to encourage researchers to share and compare their robots' performance to other systems in the database. Additionally we hope developers can find exciting robots through this benchmark, that include features they might be interested in and thus make their own innovations more efficient.

To access the benchmarking database and enter new robots please register/login HERE

General Description

As agility, in general, is highly related to the speed of how a task can be done, all scores proposed are normalized and dimensionless speeds, guaranteeing comparability between different robots and animals. Normalization and dimensionless scores are achieved by employing the scaling method of Hof [41]. As described in the following, there are thirteen scores which should, in the authors opinion, form the core concept of agility in legged terrestrial systems. These initial tasks were inspired by the ones generally visible in dog agility competitions as well as behaviors that enable the animals to react quickly to changes in their environment. Additional tasks can and may be added in the future to extend the benchmark. The higher the scores for turning (Ats and Atr), leaping (Al, Alv and Aj ), slope running (As1, As2 and As3), standing up (Ast1, Ast2), sidestepping (Asstep) as well as forward and backward locomotion (Afl and Abl) are, the better the agility is. The lowest possible score is zero. Although negative scores are possible, we disregard them as they only show how bad a system is in achieving a motion. This badnessscore may nevertheless give researchers clues for their robot improvement to reach an agility score higher than 0. To take the quality regarding precision and repeatability into account, certain variance factors will be introduced for each score. Furthermore, an overall weighted agility score as the sum of the components is proposed and also correlated with the cost of agility (COA). The scores are kept as simple as possible to allow easy experimental implementation. To provide the needed baseline for agile locomotion, we decided to, exemplary, analyze the performance of agility dogs performing different tasks during competitions and training sessions. The baseline will serve as a reference for the agility scores, enabling fair contribution of each score to the overall agility benchmark.

Measurement of Geometrical Values

To allow uniformity when defining the geometrical values for robots with different shapes and number of legs, the following scheme should be applied. Robot length lR is to be measured from the first hip axis to the last one with fully elongated body. The width wR is defined as the distance between the outer edges of two opposite legs at hip level. The last value, robot height hR is taken as distance from ground to hip-axis in an upright standing posture (leg joints extented to reach maximum leglength). The same posture is used when defining the height of the center of mass (COM), hCOM, also including the mass of the legs.

Experimental setup

Due to the simplicity of the proposed method, getting good and reliable data from the experiments does not impose the need of high technology. We propose 2 different setups, which deliver sufficient accuracy to derive the needed parameters. Table 1 gives an overview of the generally needed equipment in the minimal setup. The equipment does not include the actual slopes or other installations one would choose for the experiments.

Having a quality motion capturing system makes recording the needed data from experiments easier. Never the less it is advised, especially for illustration and comparison purposes, to record the experiments with high speed video. One camera should be mounted in top-down-view and the other one from a side view. If setup two is chosen, a scale for the respective movement has to be in the picture-frame of the camera, so the achieved movement can be quantified. The time, needed in the calculation of every agility-number, can either be extracted from the video or from the Mocap-data by counting the frames and bringing them in correlation with the respective frame-rate of the recording-system.