Can I add a device dynamically at runtime?

Yes, this is possible. In many simple and static scenarios, it is possible to add all i/o devices to the device configuration in advance, as outlined in the tutorial. However, in some scenarios, you may not know all devices beforehand, and additional devices are discovered while the dialogue is running (e.g. a user entering the room with her smart watch), or devices are removed.

For a massive multimodal system it is important that the system is aware of existing and available devices. In SiAM-dp the device manager is responsible for this, a service that keeps track of the registered, discovered, and connected devices. A device is represented in the project specification with information about the modality, the communication direction, channel information, an application internal identifier, and a physical identifier. Furthermore, the project specification provides information about which user has access to the device and in which dialogue session the device is involved.

Device Assignment

The device manager maintains three different lists for the device management:

  1. Registered Devices: These are devices that are registered in the project definition and thus are supported by the dialogue application. Additionally, new devices can be registered and existing devices can be unregistered during runtime.
  2. Discovered Devices: This is a list of discovered devices that exist in the environment and are available to the dialogue application. Devices can be either registered by the device discovery in device plugins that run inside the OSGi framework or by the specification of a TCPDevice with corresponding information about ip and port number. These can be provided directly in the project specification (or be discovered by the knowledge server: planned for a later version). Connections to TCP devices are established automatically. Devices can be discovered and lost during runtime.
  3. Connected Devices: Devices that are actually connected and involved in the dialogue application.

During runtime not every registered device must be present and not every discovered device must be connected to the dialogue application. In fact, the device manager permanently compares the list of discovered devices with the list of registered devices. If a device is discovered that meets the requirements of a registered device, the dialogue application connects to this device and makes it accessible to the dialogue. Furthermore, the device manager is responsible for creating a subscription pattern that subscribes to those messages from the event manager that are addressed to the device.

There are two stages for the matching process between registered and discovered devices. First, if the physical id of the device is given with the device registration, the assignment is explicit and only the concrete discovered device with an equal physical id matches. If no physical id is given the matching process is made by unification on the attributes given with the device descriptions, like user, modality, or communication direction. Thus it is possible to register devices to a dialogue application independently from the concrete connected devices which make devices easily exchangeable.

Device Manager Interface:

The device manager  is accessible via the OSGi service framework. It provides an interface of type IDeviceManager:

public interface IDeviceManager {
  // a device is discovered/lost by the knowledge manager or 
  // another bundle that provides a device implementation
  // these methods manipulate the list of discovered devices
 
  void deviceDiscovered(Device device);
  void deviceLost(Device device);
 
 // adds a device to the list of registered devices. 
 // The device is not necessarily available and active.
 
  registerDevice(Device device);
  unregisterDevice(String deviceId);
 
  // confirms the connection/disconnection to a device
 
  void deviceConnected(Device device);
  void deviceDisconnected(Device device);
 
  void reset();
}

Note that when using dynamic devices, it may no longer be appropriate to refer to devices by their ID when generating output. Instead, you should generate output that selects the output device for example by user and modality (e.g. you output a SpeechSynthesis act that has addressee = user1 and modality = SPEECH). Then, SiAM-dp will find a matching output device if one is available.

Even more general, you could create only modality-independent semantic output and have a generator produce the syntactic representation for those modalities which are available…

Category: General Input / Output
Tags:

← Can I add a device dynamically at runtime?