State of the Art in IoT interoperability today
These articles, since 2012, have focused on a number of architecture and design topics for IoT systems including REST API design, M2M protocol integration, and hypermedia controls. The "how" part of the problem of interaction between application software and networked things has, since 2012, been explored and implemented in many, sometimes competing, systems and standards.
Throughout, there has been an assumption that there is some underlying semantic database that informs the "what" part of the problem, the high level conceptual representation of the control affordances offered by connected things. Further, the association of these control affordances to real-world entities when a system is installed is often left to the individual IoT platforms to work out.
For example, an IoT system may provide lighting control. An application may have a control for changing the brightness of one or more light sources. The application knows "what" it wants to do, to affect the brightness of a light source. The devices that implement the light source have controls for changing brightness that conform to some data model and some protocol, which determine "how" to change the brightness of the light source.
In addition, how do I tell the application about rooms in my house, such that it knows how to control the kitchen light?
So far, interoperability between applications and connected things has been achieved by enforcing a broad agreement between all applications and all devices, both how to identify "what" is to be done, e.g. brightness control, and "how" it is to be done, e.g. changing the 16 bit integer value at software register number 3100 to set the desired brightness on a scale from 0 to 65535 representing 0 to 100% of the light source's brightness range. And so on for hundreds and thousands of settings, quantities, actuation values, across myriad IoT devices. That is to say, the operational semantics and the protocol bindings are specified by a system or standards document and agreed on by all implementers.
There are at least two major problems with this approach. One is industry fragmentation. As industry competitors squared off around IoT business pursuit, they also backed competing standards initiatives. Think of it in terms of VHS vs. Betamax, only with several competing standards backed by several different consortia. This is useful to a point, for healthy competition and technology tailored to specific use cases, but customers ultimately pay the price of incompatibility across standards. Additionally, industry growth is limited due to the perception of risk backing one standard at the expense of others.
Another limiting factor is the technical debt and brittleness associated with a monolithic approach to system design that is imposed by a "full stack" specification that contains semantic definitions as well as protocol implementation. It is often difficult to evolve the entire stack as a single entity, so some separation is inevitable. One part or the other seems to be always catching up.
Not surprisingly, the semantic definitions for connected things are all very similar. Brightness control for a light source, in the abstract, is much the same regardless of the IoT protocol used to control it. But since each of these competing IoT standards bodies had their own local context to work in, each one now has a slightly different semantic representation of light source brightness, and of course a very different protocol mapping. The IoT landscape now contains multiple regions of self-consistent, and mostly cross-consistent, but mutually incompatible digital interfaces.
What is needed is a consistent Information Model that describes the "what" in a common form that can be used across various implementation standards with diverse data models and many different ways to carry out the "how".
Toward a common Information Model for IoT
An Information Model differs from a data model mostly in level of abstraction. An information model describes concepts and relationships, where a data model describes formats and encodings. To some extent, we have used the term "data model" when we also mean "information model" but the key difference is level of abstraction.
https://tools.ietf.org/html/rfc3444 is a good document for those that want to pursue the information model discussion further.
Some research work has been going on since around 2016 on the subject of IoT Semantic Interoperability, starting with the IAB Workshop on IoT Semantic Interoperability (IOTSI) where the question was broadly taken up across industry. Some of the outcome of that workshop was to start an industry work stream to converge IoT information models and develop a common information model. We also identified the need to include development of a high level Interaction Model in the work.
We identified the concept of a meta-model for IoT information models, a small ontology derived from well-known concepts that can be used as a template for the diverse application-specific information and interaction models that diverse IoT devices require. How can we normalize the information models at the conceptual level, such that adapting to diverse operational protocols can be done in a declarative way? What is the common ground that would enable mechanical protocol translation?
In effect, we set out to create a second "narrow waist" in system design such that application software only needs to be aware of an abstract semantic interface, consisting of common semantics and interactions, and the system can take care of adapting to the various end protocols. We are creating a single interaction layer, similar to IP networking in systems, that can minimize the complexity associated with system protocols and evolving interfaces. We are in effect de-coupling the semantics from the operational protocols around this narrow waist.
Here, the application can spend its effort on "what" to do, the automation, orchestration, and behavioral aspect of the system, without regard for "how" the individual control operations are carried out. Applications can evolve separately from protocols, being decoupled by semantic APIs.
Applications can focus on integration with physical ontologies that describe rooms in the building, or parts of an automobile, or components of a heating system, and not worry that these associations will need to be re-factored when the protocols change.
IoT meta-model convergence
What is the basis for converging IoT information models? Starting with connected things, what is the common set of characteristics to which we can reduce all, or most, device models? Is there a common meta-model for IoT devices? What are the "atomic" elements of a generalized ioT information model?
These questions were taken up in follow-on activities to the IOTSI workshop of March 2016, specifically the W3C Web of Things Working Group (WoT), The Work on IoT Semantic and Hypermedia Interoperability (WISHI) in the Thing to Thing Research Group (T2TRG) of the IRTF, with iotschema.org, and most recently in the One Data Model Liaison group (OneDM) and SDF language.
Perhaps not surprisingly, the various IoT standardization and system designs do follow some consistent patterns in their information architecture.
The designs are mostly modular at the function block level. That is, the information model defines basic functional components that are reusable and composable into larger, more complex, functions. For example, a light source usually has separate information model components for on/off control, brightness control, color control, energy monitoring, and other specialized functions. This allows the same building blocks to be used in a simple on/off light as in the more complex color-controlled light.
For example, on/off control only reaches a certain level of complexity (more than many would imagine) and at some point becomes a universally applicable control to many situations. Applications only need to understand the parts they need, for example if there are timers provided they will be optional. Applications can use the same on/off control to switch lights, pumps, heaters, and so on.
Conducting a survey across a broad sweep of systems and standards reveals a common meta-model that can be extracted. The abstract information and interaction models are variously based on a small set of abstract classes that more or less align with software programming models.
Within modular functions like on/off control (e.g. software objects) there are classes that correspond to:
- State items that have retrievable and settable values
- Control entry points, like RPCs, that have input and output arguments
- Asynchronous signals that indicate something has happened.
One Data Model Liaison Group
The One Data Model liaison group was formed in late 2018, with the participants including the Zigbee Alliance, Open Connectivity Foundation (OCF), Open Mobile Alliance (OMA, LWM2M, IPSO), Bluetooth Mesh, and associated corporate representation. The initial goal was to select a set of common data models (information models) for IoT that could be shared across industry.
The work throughout 2019 and early 2020 focused on a common meta-model and representation language, known as Semantic Description Format, or SDF. In late 2020, SDF was introduced in a new IETF working group to become an internet standard. The One Data Model liaison group will use SDF to adopt and publish a common set of information models for IoT, using the meta-model described above.
The meta-model we are converging on consists of modular function definitions, classified as "Objects" in OneDM/SDF, which contain *interaction affordances* classified as Properties, Actions, and Events, and which have associated semantic information types (Data Types) with physical quantities, engineering units, and data type constraints.
SDF Feature Alignment Examples
There are currently over 200 model definitions in the OneDM "playground" for IoT device functionality, contributed mostly from OCF and OMA/IPSO Smart Object specifications. These models have been machine-converted from their native organization format. Contributed models are expected to form the basis of the OneDM published models. We are building a community process based on consensus to arrive at a set of adopted models that can be widely used in industry.
We have agreement from the contributing organizations to publish the models under the BSD 3-Clause Open Source license. This makes the models freely available for anyone to use and extend, while limiting the legal obligations of the contributors. It is expected that OneDM models will mix and match features from contributed models freely and publish models that embody the best elements and practices from all of the contributors.
OneDM will be a place where domain experts can build models specific to their applications, while taking advantage of the common and reusable tools, models, and expertise across all of the contributors. Implementers can then take these models and use them with any operational protocol they need to. The same OneDM lighting brightness model can be used with Zigbee protocols, or the OCF stack, or with OMA LWM2M protocol.
The SDF language used in OneDM is designed to be mechanically translated to and from the native SDO model formats. LWM2M XML models, OCF models using OpenAPI (Swagger), and other source formats are translated bi-directionally to and from SDF. The API mappings are maintained separately from the abstract models, and can be applied orthogonally to any SDF model. This can be used to construct translating gateways between one IoT protocol and another, by using SDF and OneDM models as and intermediary translation step between the two endpoints, as a sort of Semantic Proxy.
Practical Semantic Interoperability
Having a consistent set of abstract models provides a way to normalize the application interfaces to a variety of IoT protocols, but we need to build systems. We need to provide a consistent way to represent the semantic abstractions and the protocol bindings in a deployed system.
It's not enough to specify JSON and HTTP, for example. There are still a lot of degrees of freedom to use the standard formats and protocols in specific ways. For example, POST might be used for many different purposes in HTTP. The topic structure for an MQTT deployment can be designed in many different ways. JSON can be structured in a nearly infinite number of variations to transmit the same information.
We could rely on the written text of a standard to specify these things, but that would require bespoke implementation effort practically every time a new vendor wants to build a platform or employ a new protocol. It would be much better to have a standard, declarative, and descriptive way to specify the data formats and protocol options.
Ideally, we could extend the concepts of hypermedia controls, links and forms, to include the additional protocol metadata necessary to implement protocol bindings.
W3C Web of Things and Instance Binding
The W3C Web of Things Working Group was chartered in 2015, and has standardized a hypermedia-like format for describing instances of IoT devices and services. Thing Description (TD) is an RDF based format, using JSON-LD, that incorporates the common meta-information model of Properties, Actions, and Events, along with an extensible system of operational protocol bindings, and security protocol bindings.
Thing Description specifies a template for expressing the abstract semantic affordances of a connected thing or service, and includes protocol-level metadata for the security and operational interactions necessary to operate the thing or service within the context of some particular system.
W3C Web of Things does not specify any of the application semantics per se, only the template for a common means of expression. The Thing Description Recommendation specifies the format and vocabulary for the generic abstract affordances of Properties, Actions, and Events but does not provide any application-specific definitions for things.
The intention is for Thing Descriptions to be constructed and annotated based on application semantics from external and domain-specific ontologies and vocabularies. For example, a W3C Thing Description for a particular light bulb can be constructed based on a set of One Data Model definitions for application semantics of lighting controls (on/off, brightness, color control, energy monitor), and contain the protocol binding metadata for the specific protocols used (e.g. OMA LWM2M) such that an application that understands the abstract semantics can use a software library to operate the light bulb controls.
Further, a Semantic API can be employed such that a single application can operate any light bulb that conforms to the abstract semantics (e.g. of One Data Model) given access to the generic protocol library for the end device protocol (e.g. OMA LWM2M). The system can obtain the protocol operational instructions from the Thing Description of the device and perform operations invoked by the abstract application software. This is the mechanism that enables the Semantic Proxy described earlier.
W3C Thing Description also enables Semantic Discovery in the system, where an application is able to configure IoT things and services based on abstract specifications. For example, a lighting application may have a goal of illuminating a room with a green color. The application can search a set of Thing Descriptions to identify which light sources have color control, i.e. discovering which resources can be used to accomplish the goal. The database of Thing Descriptions would be hosted in a discovery service, using a protocol like DNS-SD or a directory lookup system like Resource Directory with a semantic search interface. W3C WoT Working Group are currently developing discovery services to meet these criteria, for publication in a follow-on Recommendation.
The W3C WoT project includes a Scripting API that provides a standard programmatic interface to devices and services exposed through WoT Thing Descriptions. There is an implementation of this scripting API and an extensible set of protocol bindings in the node-wot project on Eclipse. Node-wot is a quick way to deploy W3C WoT in a system and can implement both clients and servers, thus a WoT instance is known as a servient. Wot applications use Thing Descriptions to instantiate exposed thing instances from a node-wot server as well as consume thing instances from a node-wot client.
Another implementation of W3C WoT uses the Node-RED platform. In this implementation, Thing Descriptions are consumed by a Node Generator, which supplies a set of application-specific and instance-specific Node-RED Nodes to the developer to insert into Node-RED Flows that define the application. Additional nodes are generated that can convert values and scales and perform other adaptations.
High Level Semantic Integration - iotschema.org
Another offshoot from the 2016 IAB IoT Semantic Interoperability conference is iotschema.org. The original charter was to create extensions to schema.org for IoT devices and services.
Similar to One Data Model, iotschema definitions encapsulate some element of composable functionality, and the definitions follow the same semantic meta-model as One Data Model and W3C Thing Description, of Properties, Actions, and Events.
Definitions in iotschema.org can be easily imported and machine converted from One Data Model, which immediately makes all of the OneDM models available in RDF.
The advantage is that iotschema uses RDF and is based on RDF linking between the elements of a definition. Thus, the relationships between elements can be described using additional vocabulary, and additional relationships between individual elements and external resources can be described.
This enables iotschema definitions to be used much more flexibly than OneDM definitions. An application can make use of the entire device control apparatus, as in the case of the light bulb example above, or might only process usage data from the device, in which case the RDF links can provide the context for the data without burdening the application with needing to know how to interact with the device using the full operational semantics.
Since iotschema uses RDF, it is trivial to extend the semantics of a definition when it is used in some system, by adding additional links to the physical system the device or service is part of. Initially, we provide a Feature of Interest (FoI) link property to deploy the FoI pattern from the Observation and Measurement ontology, also used in SSN and SAREF. FoI links are included to link the physical context of an IoT device or service, for example to indicate that a sensor measures air temperature in a HVAC return duct from the building atrium.
Likewise, iotschema definitions, in system context, can incorporate extended data semantics to describe engineering types, data provenance, quantities and units, and arbitrary classifiers, e.g. information from a MUD.
Extension semantics use existing and emerging ontologies. For example, there are existing IoT ontologies like SAREF and SOSA that can be cross-linked with iotschema. In W3C there are domain specific ontologies, for example VSSschema for automotive, and BRICKschema for connected and intelligent buildings. There is ongoing work to incorporate geospatial information models, to integrate location and position information.
While we are not at this time proposing any specific extensions for schema.org proper, we are advancing co-operation with the W3C Web of Things Interest Group (WoT IG) and actively participating in W3C WoT Plugfests by providing application models and application-specific vocabularies for common IoT functions.
Summary
We have developed IoT Semantic Interoperability in a number of separate fora, converging around common emergent features of industry semantic meta-models.
- IAB IOTSI - initial conference in 2016 to map out the problem space and establish a baseline from existing work
- IRTF WISHI - ongoing activity to develop practical semantic interoperability through specification and implementation
- W3C WoT - ongoing W3C standards track for a "web language" for standardized interactions with connected things
- iotschema.org - Application-specific models in RDF that conform to a standardized meta-model with shape constraints, well-known integration point with other ontologies
- One Data Model Liaison Group - Liaison of SDOs and industry experts to create a unified set of application-specific models for IoT devices and services
- IETF ASDF Working Group - Specification of a standardized and developer-friendly "language" for application-specific semantic models for IoT devices and services
The basis for semantic interoperability is a common semantic meta-model, with associated application-specific and vertical extensions, that all applications can use as a semantic template for high level abstract interactions with a wide variety of real-world devices.
In practice, semantic interoperability requires a system of Protocol Binding that enables the abstract models and definitions to be associated with a particular implementation of the abstract model. This can be thought of like an extended hypermedia control. The processing of these can be at system build time or at runtime, depending on the need for dynamic adaptation.
Practical semantic interoperability for IoT also requires semantic connections to the real world, to enable the affordances offered by devices, for example, light source brightness, to be associated with real world features, such as rooms in a house or areas in a public space.
A Semantic API enables application developers to limit their concerns to the high level abstract functions being performed, and avoid issues with protocol conversions and adaptation. Common meta-models and shared application models enable truly portable applications for IoT to be developed.
Much of the underlying software to accomplish this exists, including model translators from SDF to and from SDO data models like OCF and LWM2M, Runtime systems like node-wot and WoT for Node-RED, and tools developed in One Data Model and iotschema.org to provide developer-friendly infrastructure. We have been testing the integration patterns in W3C WoT Plugfests since 2016.
Next steps
Going forward, we plan to continue to integrate and develop the common meta-model across more of industry and more deeply across the core stack technologies.
We are calling for participation by SDOs and vertical industry groups in One Data Model, to develop industry-specific models and harmonize around the common meta-model and tools. One Data Model will begin publishing "adopted models" that are guaranteed to be stable and maintained through the community.
In iotschema.org, we will work with W3C Web of Things and other domain-specific groups on semantic integration.
W3C Web of Things development is focused on discovery and protocol binding development, and consolidating architecture and use cases. WoT Profile is an emerging standard for simple interoperability between devices and applications.
The IETF ASDF Working Group expect to publish an RFC for the SDF language later this year, and will continue to develop SDF to address more use cases.
About the Author
Michael J Koster is a system architect and engineer with over 30 years experience designing large scale, complex, and embedded systems. He has chaired and co-chaired various IoT standards activities relating to IoT Data Models and Semantic Interoperability, including IPSO Smart Objects, OCF Data Models, iotschema.org, and most recently the One Data Model Liaison Group. His background includes computer architecture, performance modeling, hardware design, energy systems, process control systems, and analog electronics. He is currently active in IETF and W3C WoT groups.
