Data Flow Programming

Adaptive Vision Studio is a drag and drop environment designed for machine vision professionals. You don't have to be a programmer to use it. Nevertheless, you have to understand its 4 core concepts: Data, Filters, Connections and Macrofilters. This section explains them in detail.

Data

Adaptive Vision Studio is about data processing. There are some data on the program input (usually images) and through a sequence of data processing steps they are transformed into meaningful outputs (e.g. into a pass/fail information).

Types and Values

The most important thing about data that has to be understood is the distinction between types and values. In Adaptive Vision Studio the former are important during program construction as they define which inputs and outputs can be connected. The latter appear only during program execution and represent information that is processed.

More information:

Type Sample Values
Integer 5, 19, -128
String "Intuitive", "Powerful", "Adaptable"
Image
,

Arrays

Multiple data of the same type can be grouped and processed together. In Adaptive Vision Studio such aggregates are called arrays. For each type, there is a corresponding array type. For example just as 4 is a value of type Integer, collection {1, 5, 4} is a value of type IntegerArray. In real-life applications you will often find arrays of numbers, points, regions etc. Furthermore, arbitrarily nested arrays of arrays are also possible. For example IntegerArrayArray is a type of multiple arrays of integers.

Read also about Arrays in User Manual.

Filters

Filters are basic data processing elements from which programs are constructed. Each filter can have multiple inputs and outputs of different types. Inputs can be connected to outputs of other filters or can be set to constant values in the properties window.

Filters operate in 3-step cycles:

  1. Read input data.
  2. Execute data processing.
  3. Produce output data.

State

Most filters in Adaptive Vision Studio are purely functional - for the same input they always produce the same output. There are however some filters that behave differently - what they produce depends not only on the current input data, but also on an internal state, which stores information between consecutive executions of the filter. Such filters are commonly used in hardware suppor, video processing and for computing of aggregate results.

For example all filters with _OfTheLoop suffix perform some operation on a sequence of values appearing in consecutive iterations. State is used to remember intermediate results.

Generic Filters

Some filters are required for many different types - for example the ArraySize filter should work for arrays of integers, images and even for arrays of arrays. For that reason in Adaptive Vision Studio the types of its ports can be adapted. In the Filter Catalog you will see a special type T, which you will be able to change when you drop an instance into a program editor.

Connections

Introduction

In general, connections in a program can be created between inputs and outputs of compatible types. That does not mean however that the types must always be equal. You can connect ports of different types if only it is possible to adapt transferred data values by means of automatic conversions, array decompositions, loops or conditions. The power of this drag & drop programming model stems from the fact that a user just drags a connection and all the logic is added implicitly in a "do what I mean" way.

Connection Logic

This is probably the most complicated part of the Adaptive Vision programming model as it concentrates most of its computing flexibility. The good news is that you will never define any connection logic explicitly as this is done by the application automatically.

There 5 kinds of connection logic:

  1. Basic connection
    - this kind of connection appears between ports of equal types.

  2. Automatic conversion
    - automatic conversion is used when a user connects ports of two different types, e.g. Integer and Real, and there is an appropriate filter in the Conversions category, e.g. IntegerToReal, that can be used to translate the values.

  3. Array connection
    - this logic creates an implicit loop on a connection from an array to a scalar type, e.g. from RegionArray to Region. All the individual results are then merged into arrays and so the outputs of the second filter turn into arrays as well. See also: Arrays in the User Manual.

  4. Singleton connection
    - conversely, when an output of a scalar type is connected to an array input, e.g. Point2D to Point2DArray, the singleton connection assures that a value will be converted into a single element array.

  5. Conditional connection
    - the last kind, a conditional connection appears when conditional values (T?) are transferred to a non-conditional input (T). This causes the second filter to be executed conditionally depending on whether a value exists or not (then it is NIL). Its outputs change into conditional outputs accordingly, assuring that consecutive filters will be executed conditionally as well. Conditional execution ends at a filter with a conditional input. More information on this topic can be found in Conditional Execution section of the User Manual.

Furthermore, for maximum flexibility many combinations of the above logics can appear.

Show all connections.

  1. Array connection with conversion
    - all elements are individually converted.

  2. Array connection with conditional elements
    - output arrays are comprised only from elements corresponding to non-NIL input elements.

  3. Array connection with conditional elements and conversions
    - as above plus conversions.

  4. Singleton connection with conversion
    - a scalar is converted into a single element array after a conversion.

  5. Conditional array connection
    - all array elements are processed or none depending on a condition.

  6. Conditional connection with conversion

  7. Conditional array connection with conversion
    - all array elements are processed and converted individually or none depending on a condition.

  8. Conditional singleton connection

  9. Conditional singleton connection with conversion

Macrofilters

Macrofilters

Several filters configured and connected in an appropriate way make up a simple program. Most often there is an image acquisition filter at the beginning and then several image processing steps. Such programs run in a loop and typically retrieve some information from consecutive input frames.

However, for bigger real-world applications it is necessary to create several subprograms corresponding to different processing steps. In Adaptive Vision Studio these are called macrofilters and differ from whole programs only in that they can have their own inputs and outputs. After being created a macrofilter appears in This Program category in the Filter Catalog and since then can be used exactly in the same drag & drop way as normal filters. This significantly reduces the overall complexity of a program structure.

Macrofilter Iterations

Macrofilters are in fact not only a means to organize big programs. What is equally important, each macrofilter can perform several iterations before it produces its output values and quits.

To enable iterations insert at least one generator filter to the macro. The most obvious example of a generator is the GrabImage filter. It generates consecutive images from a camera in consecutive iterations, forever. Another example of a generator is the EnumerateIntegers filter, which generates consecutive integer numbers. This one can be set to terminate at some number and thus to make the macro it is in finish computations. More specifically, a macrofilter terminates exactly after any generator terminates. That means a termination always occurs in a middle of a macro after some filters are executed and possibly before all are executed.

See also: Program Execution in the User Manual.

It is also possible to explicitly pass information from one iteration to another - see Loop Registers in the User Manual.