Sunday, April 11, 2021

Beyond Hypermedia - Information Models and Semantic Interoperability

State of the Art in IoT interoperability today

These articles, since 2012, have focused on a number of architecture and design topics for IoT systems including REST API design, M2M protocol integration, and hypermedia controls. The "how" part of the problem of interaction between application software and networked things has, since 2012, been explored and implemented in many, sometimes competing, systems and standards. 

Throughout, there has been an assumption that there is some underlying semantic database that informs the "what" part of the problem, the high level conceptual representation of the control affordances offered by connected things. Further, the association of these control affordances to real-world entities when a system is installed is often left to the individual IoT platforms to work out.

For example, an IoT system may provide lighting control. An application may have a control for changing the brightness of one or more light sources. The application knows "what" it wants to do, to affect the brightness of a light source. The devices that implement the light source have controls for changing brightness that conform to some data model and some protocol, which determine "how" to change the brightness of the light source. 

In addition, how do I tell the application about rooms in my house, such that it knows how to control the kitchen light?

So far, interoperability between applications and connected things has been achieved by enforcing a broad agreement between all applications and all devices, both how to identify "what" is to be done, e.g. brightness control, and "how" it is to be done, e.g. changing the 16 bit integer value at software register number 3100 to set the desired brightness on a scale from 0 to 65535 representing 0 to 100% of the light source's brightness range. And so on for hundreds and thousands of settings, quantities, actuation values, across myriad IoT devices. That is to say, the operational semantics and the protocol bindings are specified by a system or standards document and agreed on by all implementers.

There are at least two major problems with this approach. One is industry fragmentation. As industry competitors squared off around IoT business pursuit, they also backed competing standards initiatives. Think of it in terms of VHS vs. Betamax, only with several competing standards backed by several different consortia. This is useful to a point, for healthy competition and technology tailored to specific use cases, but customers ultimately pay the price of incompatibility across standards. Additionally, industry growth is limited due to the perception of risk backing one standard at the expense of others.

Another limiting factor is the technical debt and brittleness associated with a monolithic approach to system design that is imposed by a "full stack" specification that contains semantic definitions as well as protocol implementation. It is often difficult to evolve the entire stack as a single entity, so some separation is inevitable. One part or the other seems to be always catching up.

Not surprisingly, the semantic definitions for connected things are all very similar. Brightness control for a light source, in the abstract, is much the same regardless of the IoT protocol used to control it. But since each of these competing IoT standards bodies had their own local context to work in, each one now has a slightly different semantic representation of light source brightness, and of course a very different protocol mapping. The IoT landscape now contains multiple regions of self-consistent, and mostly cross-consistent, but mutually incompatible digital interfaces.

What is needed is a consistent Information Model that describes the "what" in a common form that can be used across various implementation standards with diverse data models and many different ways to carry out the "how".

Toward a common Information Model for IoT

An Information Model differs from a data model mostly in level of abstraction. An information model describes concepts and relationships, where a data model describes formats and encodings. To some extent, we have used the term "data model" when we also mean "information model" but the key difference is level of abstraction. is a good document for those that want to pursue the information model discussion further.

Some research work has been going on since around 2016 on the subject of IoT Semantic Interoperability, starting with the IAB Workshop on IoT Semantic Interoperability (IOTSI) where the question was broadly taken up across industry. Some of the outcome of that workshop was to start an industry work stream to converge IoT information models and develop a common information model. We also identified the need to include development of a high level Interaction Model in the work. 

We identified the concept of a meta-model for IoT information models, a small ontology derived from well-known concepts that can be used as a template for the diverse application-specific information and interaction models that diverse IoT devices require. How can we normalize the information models at the conceptual level, such that adapting to diverse operational protocols can be done in a declarative way? What is the common ground that would enable mechanical protocol translation?

In effect, we set out to create a second "narrow waist" in system design such that application software only needs to be aware of an abstract semantic interface, consisting of common semantics and interactions, and the system can take care of adapting to the various end protocols. We are creating a single interaction layer, similar to IP networking in systems, that can minimize the complexity associated with system protocols and evolving interfaces. We are in effect de-coupling the semantics from the operational protocols around this narrow waist.

Semantic Narrow Waist

Here, the application can spend its effort on "what" to do, the automation, orchestration, and behavioral aspect of the system, without regard for "how" the individual control operations are carried out. Applications can evolve separately from protocols, being decoupled by semantic APIs. 

Applications can focus on integration with physical ontologies that describe rooms in the building, or parts of an automobile, or components of a heating system, and not worry that these associations will need to be re-factored when the protocols change.

IoT meta-model convergence

What is the basis for converging IoT information models? Starting with connected things, what is the common set of characteristics to which we can reduce all, or most, device models? Is there a common meta-model for IoT devices? What are the "atomic" elements of a generalized ioT information model?

These questions were taken up in follow-on activities to the IOTSI workshop of March 2016, specifically the W3C Web of Things Working Group (WoT), The Work on IoT Semantic and Hypermedia Interoperability (WISHI) in the Thing to Thing Research Group (T2TRG) of the IRTF, with, and most recently in the One Data Model Liaison group (OneDM) and SDF language.

Perhaps not surprisingly, the various IoT standardization and system designs do follow some consistent patterns in their information architecture. 

The designs are mostly modular at the function block level. That is, the information model defines basic functional components that are reusable and composable into larger, more complex, functions. For example, a light source usually has separate information model components for on/off control, brightness control, color control, energy monitoring, and other specialized functions. This allows the same building blocks to be used in a simple on/off light as in the more complex color-controlled light. 

For example, on/off control only reaches a certain level of complexity (more than many would imagine) and at some point becomes a universally applicable control to many situations. Applications only need to understand the parts they need, for example if there are timers provided they will be optional. Applications can use the same on/off control to switch lights, pumps, heaters, and so on.

Conducting a survey across a broad sweep of systems and standards reveals a common meta-model that can be extracted. The abstract information and interaction models are variously based on a small set of abstract classes that more or less align with software programming models.

Survey of Semantic Categories from various IoT Standards

Within modular functions like on/off control (e.g. software objects) there are classes that correspond to: 

  • State items that have retrievable and settable values
  • Control entry points, like RPCs, that have input and output arguments
  • Asynchronous signals that indicate something has happened.

One Data Model Liaison Group

The One Data Model liaison group was formed in late 2018, with the participants including the Zigbee Alliance, Open Connectivity Foundation (OCF), Open Mobile Alliance (OMA, LWM2M, IPSO), Bluetooth Mesh, and associated corporate representation. The initial goal was to select a set of common data models (information models) for IoT that could be shared across industry. 

The work throughout 2019 and early 2020 focused on a common meta-model and representation language, known as Semantic Description Format, or SDF. In late 2020, SDF was introduced in a new IETF working group to become an internet standard. The One Data Model liaison group will use SDF to adopt and publish a common set of information models for IoT, using the meta-model described above.

The meta-model we are converging on consists of modular function definitions, classified as "Objects" in OneDM/SDF, which contain *interaction affordances* classified as Properties, Actions, and Events, and which have associated semantic information types (Data Types) with physical quantities, engineering units, and data type constraints.

SDF (One Data Model)

SDF Feature Alignment Examples

There are currently over 200 model definitions in the OneDM "playground" for IoT device functionality, contributed mostly from OCF and OMA/IPSO Smart Object specifications. These models have been machine-converted from their native organization format. Contributed models are expected to form the basis of the OneDM published models. We are building a community process based on consensus to arrive at a set of adopted models that can be widely used in industry.

We have agreement from the contributing organizations to publish the models under the BSD 3-Clause Open Source license. This makes the models freely available for anyone to use and extend, while limiting the legal obligations of the contributors. It is expected that OneDM models will mix and match features from contributed models freely and publish models that embody the best elements and practices from all of the contributors.

OneDM will be a place where domain experts can build models specific to their applications, while taking advantage of the common and reusable tools, models, and expertise across all of the contributors. Implementers can then take these models and use them with any operational protocol they need to. The same OneDM lighting brightness model can be used with Zigbee protocols, or the OCF stack, or with OMA LWM2M protocol. 

The SDF language used in OneDM is designed to be mechanically translated to and from the native SDO model formats. LWM2M XML models, OCF models using OpenAPI (Swagger), and other source formats are translated bi-directionally to and from SDF. The API mappings are maintained separately from the abstract models, and can be applied orthogonally to any SDF model. This can be used to construct translating gateways between one IoT protocol and another, by using SDF and OneDM models as and intermediary translation step between the two endpoints, as a sort of Semantic Proxy.

Semantic Proxy

Practical Semantic Interoperability

Having a consistent set of abstract models provides a way to normalize the application interfaces to a variety of IoT protocols, but we need to build systems. We need to provide a consistent way to represent the semantic abstractions and the protocol bindings in a deployed system. 

It's not enough to specify JSON and HTTP, for example. There are still a lot of degrees of freedom to use the standard formats and protocols in specific ways. For example, POST might be used for many different purposes in HTTP. The topic structure for an MQTT deployment can be designed in many different ways. JSON can be structured in a nearly infinite number of variations to transmit the same information. 

We could rely on the written text of a standard to specify these things, but that would require bespoke implementation effort practically every time a new vendor wants to build a platform or employ a new protocol. It would be much better to have a standard, declarative, and descriptive way to specify the data formats and protocol options.

Ideally, we could extend the concepts of hypermedia controls, links and forms, to include the additional protocol metadata necessary to implement protocol bindings.

W3C Web of Things and Instance Binding

The W3C Web of Things Working Group was chartered in 2015, and has standardized a hypermedia-like format for describing instances of IoT devices and services. Thing Description (TD) is an RDF based format, using JSON-LD, that incorporates the common meta-information model of Properties, Actions, and Events, along with an extensible system of operational protocol bindings, and security protocol bindings.

Thing Description specifies a template for expressing the abstract semantic affordances of a connected thing or service, and includes protocol-level metadata for the security and operational interactions necessary to operate the thing or service within the context of some particular system.

W3C Web of Things does not specify any of the application semantics per se, only the template for a common means of expression. The Thing Description Recommendation specifies the format and vocabulary for the generic abstract affordances of Properties, Actions, and Events but does not provide any application-specific definitions for things.

The intention is for Thing Descriptions to be constructed and annotated based on application semantics from external and domain-specific ontologies and vocabularies. For example, a W3C Thing Description for a particular light bulb can be constructed based on a set of One Data Model definitions for application semantics of lighting controls (on/off, brightness, color control, energy monitor), and contain the protocol binding metadata for the specific protocols used (e.g. OMA LWM2M) such that an application that understands the abstract semantics can use a software library to operate the light bulb controls. 

Further, a Semantic API can be employed such that a single application can operate any light bulb that conforms to the abstract semantics (e.g. of One Data Model) given access to the generic protocol library for the end device protocol (e.g. OMA LWM2M). The system can obtain the protocol operational instructions from the Thing Description of the device and perform operations invoked by the abstract application software. This is the mechanism that enables the Semantic Proxy described earlier.

W3C Thing Description also enables Semantic Discovery in the system, where an application is able to configure IoT things and services based on abstract specifications. For example, a lighting application may have a goal of illuminating a room with a green color. The application can search a set of Thing Descriptions to identify which light sources have color control, i.e. discovering which resources can be used to accomplish the goal. The database of Thing Descriptions would be hosted in a discovery service, using a protocol like DNS-SD or a directory lookup system like Resource Directory with a semantic search interface. W3C WoT Working Group are currently developing discovery services to meet these criteria, for publication in a follow-on Recommendation.

The W3C WoT project includes a Scripting API that provides a standard programmatic interface to devices and services exposed through WoT Thing Descriptions. There is an implementation of this scripting API and an extensible set of protocol bindings in the node-wot project on Eclipse. Node-wot is a quick way to deploy W3C WoT in a system and can implement both clients and servers, thus a WoT instance is known as a servient. Wot applications use Thing Descriptions to instantiate exposed thing instances from a node-wot server as well as consume thing instances from a node-wot client.

Another implementation of W3C WoT uses the Node-RED platform. In this implementation, Thing Descriptions are consumed by a Node Generator, which supplies a set of application-specific and instance-specific Node-RED Nodes to the developer to insert into Node-RED Flows that define the application. Additional nodes are generated that can convert values and scales and perform other adaptations. 

High Level Semantic Integration - 

Another offshoot from the 2016 IAB IoT Semantic Interoperability conference is The original charter was to create extensions to for IoT devices and services. 

Similar to One Data Model, iotschema definitions encapsulate some element of composable functionality, and the definitions follow the same semantic meta-model as One Data Model and W3C Thing Description, of Properties, Actions, and Events.

iotschema Pattern

Definitions in can be easily imported and machine converted from One Data Model, which immediately makes all of the OneDM models available in RDF.

The advantage is that iotschema uses RDF and is based on RDF linking between the elements of a definition. Thus, the relationships between elements can be described using additional vocabulary, and additional relationships between individual elements and external resources can be described. 

This enables iotschema definitions to be used much more flexibly than OneDM definitions. An application can make use of the entire device control apparatus, as in the case of the light bulb example above, or might only process usage data from the device, in which case the RDF links can provide the context for the data without burdening the application with needing to know how to interact with the device using the full operational semantics.

Since iotschema uses RDF, it is trivial to extend the semantics of a definition when it is used in some system, by adding additional links to the physical system the device or service is part of. Initially, we provide a Feature of Interest (FoI) link property to deploy the FoI pattern from the Observation and Measurement ontology, also used in SSN and SAREF. FoI links are included to link the physical context of an IoT device or service, for example to indicate that a sensor measures air temperature in a HVAC return duct from the building atrium.

Likewise, iotschema definitions, in system context, can incorporate extended data semantics to describe engineering types, data provenance, quantities and units, and arbitrary classifiers, e.g. information from a MUD. 

Extension semantics use existing and emerging ontologies. For example, there are existing IoT ontologies like SAREF and SOSA that can be cross-linked with iotschema. In W3C there are domain specific ontologies, for example VSSschema for automotive, and BRICKschema for connected and intelligent buildings. There is ongoing work to incorporate geospatial information models, to integrate location and position information.

While we are not at this time proposing any specific extensions for proper, we are advancing co-operation with the W3C Web of Things Interest Group (WoT IG) and actively participating in W3C WoT Plugfests by providing application models and application-specific vocabularies for common IoT functions.


We have developed IoT Semantic Interoperability in a number of separate fora, converging around common emergent features of industry semantic meta-models. 

  • IAB IOTSI - initial conference in 2016 to map out the problem space and establish a baseline from existing work
  • IRTF WISHI - ongoing activity to develop practical semantic interoperability through specification and implementation
  • W3C WoT - ongoing W3C standards track for a "web language" for standardized interactions with connected things
  • - Application-specific models in RDF that conform to a standardized meta-model with shape constraints, well-known integration point with other ontologies
  • One Data Model Liaison Group - Liaison of SDOs and industry experts to create a unified set of application-specific models for IoT devices and services
  • IETF ASDF Working Group - Specification of a standardized and developer-friendly "language" for application-specific semantic models for IoT devices and services

The basis for semantic interoperability is a common semantic meta-model, with associated application-specific and vertical extensions, that all applications can use as a semantic template for high level abstract interactions with a wide variety of real-world devices.

In practice, semantic interoperability requires a system of Protocol Binding that enables the abstract models and definitions to be associated with a particular implementation of the abstract model. This can be thought of like an extended hypermedia control. The processing of these can be at system build time or at runtime, depending on the need for dynamic adaptation.

Practical semantic interoperability for IoT also requires semantic connections to the real world, to enable the affordances offered by devices, for example, light source brightness, to be associated with real world features, such as rooms in a house or areas in a public space.

A Semantic API enables application developers to limit their concerns to the high level abstract functions being performed, and avoid issues with protocol conversions and adaptation. Common meta-models and shared application models  enable truly portable applications for IoT to be developed.

Much of the underlying software to accomplish this exists, including model translators from SDF to and from SDO data models like OCF and LWM2M, Runtime systems like node-wot and WoT for Node-RED, and tools developed in One Data Model and to provide developer-friendly infrastructure. We have been testing the integration patterns in W3C WoT Plugfests since 2016.

Next steps

Going forward, we plan to continue to integrate and develop the common meta-model across more of industry and more deeply across the core stack technologies.

We are calling for participation by SDOs and vertical industry groups in One Data Model, to develop industry-specific models and harmonize around the common meta-model and tools. One Data Model will begin publishing "adopted models" that are guaranteed to be stable and maintained through the community.

In, we will work with W3C Web of Things and other domain-specific groups on semantic integration. 

W3C Web of Things development is focused on discovery and protocol binding development, and consolidating architecture and use cases. WoT Profile is an emerging standard for simple interoperability between devices and applications.

The IETF ASDF Working Group expect to publish an RFC for the SDF language later this year, and will continue to develop SDF to address more use cases.

About the Author

Michael J Koster is a system architect and engineer with over 30 years experience designing large scale, complex, and embedded systems. He has chaired and co-chaired various IoT standards activities relating to IoT Data Models and Semantic Interoperability, including IPSO Smart Objects, OCF Data Models,, and most recently the One Data Model Liaison Group. His background includes computer architecture, performance modeling, hardware design, energy systems, process control systems, and analog electronics. He is currently active in IETF and W3C WoT groups.


IAB IOTSI Workshop


One Data Model Liaison Group

IETF ASDF Working Group

W3C Web of Things



Domain and Other Ontologies

Monday, April 3, 2017

Interactive Hypermedia and Asynchronous Machine Interaction using Hypermedia Controls


The most recent articles discussed some issues and design patterns  for machine interaction using hypermedia controls, and presented an experimental open source software platform and demonstrator for an example machine-to-machine hypermedia system.

This article will summarize the progress on this topic over the last year and focus on the subject of asynchronous and realtime interactions using REST design principles and, in particular, hypermedia controls.

HSML: Hypermedia Content Formats for Machine Interaction

Since the last article, there is a new Internet Draft describing a content format for hypermedia collections on constrained networks. HSML is a simple representation and interaction model for collections and associated links, forms, and items, using CoRE Link-Format and SenML.

Reusable Content Format

HSML is an updated version of the content format described in the earlier articles, and is kept aligned with the current versions of SenML and CoRE Link-Format.

HSML representations include CoRE Link-Format and SenML, with some extensions to the Link-Format and SenML keyword vocabularies. In addition, there is a format which includes both Link-Format and SenML elements, to facilitate interactions with items and their link parameters in a collection atomically.

HSML provides for use of the Interface Type parameter (if) in links and request parameters, and is used to select representations of a resource as defined in CoRE Interfaces.

Common Transfer Layer

The HSML draft discusses a common abstract transfer layer using CRUD + Observe/Notify, also known as CRUD+N, which is used to map various concrete protocols, including HTTP, CoAP, and MQTT.

This supports a common model for resource state interaction, upon which a consistent hypermedia interaction model may be constructed, and from there be extended.

CRUD, REST, and Hypermedia

This discussion is based on REST system design in the context of distributed state machine design.

In REST system design, a distributed system is represented as a state transition graph. The nodes of the graphs represent states of the system, and the arcs of the graph represent possible state transitions.

The common resource design pattern known as CRUD exposes the system state contained in the nodes of the graph. Resource state may be Created, Retrieved, Updated, or Deleted.

Hypermedia controls expose the system state transition model at the edges of the graph. Hypermedia controls are the hyperlinks and submission forms, or their machine equivalents, that describe the available state transitions offered by the system, and how, from the client, they may be effected.
Hypermedia Controls Describe the Available State Transitions
To bring in another related set of concepts, hypermedia controls implement a set of "control plane" abstractions that work together with the "data plane" abstraction of CRUD, forming two aspects of a distributed system architecture.

Without hypermedia controls, CRUD clients and servers will generally need to be more tightly coupled. These systems tend to use early binding of clients to servers, with schemas and introspection, code generation, and related techniques to drive client state machines to consume resources.

With hypermedia controls to dynamically describe state transitions, REST clients can consume the hypermedia controls in order to understand how to drive system state transitions, in a more loosely coupled, separately evolvable, late binding way.

Hypermedia controls describe possible state transitions based on the current state of the system, rather than pre-determined by a static resource design.

Interactive and Reactive Hypermedia: Actions and Link Bindings

In the last article, a class of hypermedia controls was described that facilitates asynchronous and realtime interaction between hypermedia clients and REST servers. These belong to a general class of interactive, and reactive, Hypermedia controls, that can direct the transfer and processing of dynamically changing data and asynchronous events.

This article describes two types of this class of Interactive Hypermedia controls, Actions and Link Bindings, in more depth, with examples and terminology consistent with the HSML Internet Draft published in the IRTF Thing to Thing research group (T2TRG):

Problem statement and scope

Asynchronous interaction over REST involves use cases for asynchronous state transfer in both directions; from client to server, and from server to client.

In the client to server case, the problem is in describing incoming state transitions which may or may not have immediate results. This enables the client to interactively direct the server processing of actions through the REST model.

In the server to client case, the problem includes describing how to use asynchronous communication to communicate resource state changes outgoing, in real time, to clients in order to enable reactive client processing of server state changes and events.

A third case is when a server device is to be controlled by another device, which also exposes a server role. The problem in this case is how to describe the client role involved in state transfer between two or more server resources.

Actions and Forms

Incoming (from the client to the server) state transitions are state transitions that a client wishes to make on a resource, such as turning a power switch on or opening a garage door.

For incoming state transitions, the problem is how to inform the client what transitions are available and how to construct requests which change the resource state of the system in some way that may be more or less indirectly related to the control input. It is not always a simple and immediate CRUD state update. The state of the resource may need to be asynchronously monitored to complete the client state transition at some time in the future.

For example, an actuator may take some time to complete its action, like closing a motorized gate or garage door. The outcome may be uncertain, as in the case of power interruptions or mechanical blockage of the gate. These conditions result in a number of possible eventual outcomes which need to be returned to the client for processing and client state update.

A meta-model using the concept of an "action" is presented in the HSML draft. An Action describes and accepts a representation for a desired state change to a resource. The conceptual model is that the desired state change is "created" in the context of an Action resource, which then affects the state of the linked resource(s), and which can then be monitored for its eventual outcome.

An Action link is indicated by the value of "rel" including "action", and works in analogous fashion to a form in HTML; it describes what the action is expected to accomplish, like "turn on the light" or "add a post to the blog", and includes information on how to accomplish the action, like what method to use, which resource type, which content format, schema information to describe the payload, and descriptions of expected responses.

Action links can describe simple action operations that use REST state updates, or more complex actions which create representations of abstract action descriptions in collections of action instances. Such collections may be used to sequence submitted actions in FIFO or priority order, and for clients to track long running or pending actions which have not yet completed.

Actions may be performed directly on resources they are intended to update the state of, or they may be performed on proxy resources, as in the likely example of action instances crreated in a collection.

In the case that Actions are performed on a proxy resource, the resource may be indicated in the context of the affected resource(s) by including a link with the relation type including the value "alternate" and with a resource type indicating the type of action.

Link Bindings

For outgoing (from the server to the client) state transitions, there is the additional problem of describing how a client is expected to asynchronously obtain the state transitions as they occur, perhaps with some additional QoS parameters specified as system constraints.

The concept of a Link Binding is presented in the HSML draft. A link binding, in this context and in the context of the CoRE dynlink Internet Draft, is a hyperlnk that defines a dynamic data update from one resource to another.

The description of link bindings assumes that there is a generalized REST-hook mechanism in place for the source of resource state updates, which can be used to trigger the link binding transfer operation. CoAP Observe may be used if available, or HTTP EventSource, or MQTT Subscribe.
Link Binding
The link source and destination are generalized to allow resource state to be communicated using an extensible set of transfer protocols. For example, links using the "mqtt:" scheme will instruct the system to publish updates  from the linked resource to an mqtt broker, or subscribe to an mqtt broker to obtain state updates for the linked resource.

The link binding can be located with the information source and cause information to be pushed or published to the destination resource, or it may be located with the destination resource and cause the source resource to be subscribed, observed, or polled.
Asynchronous Notification using the "monitor" relation
The link binding may be in fact stored and operated in a third place, separately from the source or destination, and mat use an agent at the third party to effect the source and destination transfer operations.

Link bindings may use different relation types to characterize transfer direction. In the examples above, the "monitor" relation type is used to describe a target resouce (monitor) which can be used to be informed of state changes in the context (monitored) resource. Another relation type is "boundto", as described in CoRE dynlink, which defines state transfer from the target resource to the context.

Link bindings may be defined with separate source and destination controls for transfer layer parameters like methods, content formats, and resource type queries. This enables link bindings to be generally used for resource state transfer, for triggering Actions, or for generating abstract events.

Link bindings may have different source and destination schemes, enabling link bindings to be used to convert protocols. For example, updates to a REST resource may be published to an MQTT broker using a link binding. Since the content format and other resource information can be obtained using HTTP, there is less need to augment the MQTT system with meta-data when used in this way.

For schemes that don't offer subscribe or observe, for example HTTP, RESThooks can be created and used in a structured way. Similarly, for systems using MQTT subscribe or CoAP Observe, link bindings enable the orchestration and inspection of dynamic resource interactions using graph techniques and tools.

Link bindings provide a means for conditional notification, using parameters to control the time period and value/change threshold for notification messages to be generated.

An extension to link bindings may be created which enables a link binding to consume a hypermedia action. This would enable cross-protocol adaptation without proxies, by using dynamic adaptation code in the libraries that consume the hypermedia controls.

System architecture examples

Interactive hypermedia controls enable the dynamic orchestration of client-server interactions by discovering actions and adding dynamic link bindings to connect resources, event sources, and event handlers. The following are some examples of system-level orchestration using interactive hypermedia.

Actions in hypermedia clients

The Action control enables a generic client to adapt to the state machine of the server when performing actuation and invoking actions. The client consumes the hypermedia control, using the information obtained to select an action, marshal its parameters, and apply the action to the resource using the correct method and content format.

Typically, the hypermedia control will be annotated with semantic identifiers to indicate the type(s) of action offered and the types of parameters expected, as well as any special response that may be returned.

Resouce servers may offer action design patterns optimized for the particular situation, and clients may adapt to a variety of patterns without a-priori knowledge, using hypermedia action controls.

Actions are indicated with a link relation "rel=action", as per the HSML draft.

In the examples below, a hypermedia application only needs to know the semantic meanings of the terms. The hypermedia client can adapt to the different server designs based on consuming the hypermedia action control.

Client wants the light to move to 77% brightness with a ramp time of 10 seconds

action = moveto ({targetbrightness = 77, units = %}, {ramptime = 10, units = seconds})

The client would perform a link query to discover a moveto action and verify that it know how to express the target parameters in the available units.

In this example, the client is looking for a brightness control with a "moveto" action type

GET /example/light/?if=core.ll

        "href": "",
        "rel": "self",
        "rt": "urn:example:luminaire",
        "content-format": 50,
        "if": "core.ll"
        "href": "brightness/",
        "rt": "urn:example:brightness",
        "content-format": 50,
        "if": ["core.ll", "core.b"]
        "anchor": "brightness/",
        "href": "moveto/",
        "rel": ["alternate", "action"],
        "rt": "urn:example:moveto",
        "method": "create",
        "accept-format": 50,
        "if": "core.b"

The resource at /example/light/moveto is an alternate resource for the collection at /example/light/brightness and offers action capability. The resource type (rt) is "moveto" which for action resources indicates the type of action.

Seeing the readable parameter interface on the brightness, the client could check the units and adapt if needed.

GET /example/light/brightness/?if=core.ll

        "href": "",
        "rel": "self",
        "rt": "urn:example:brightness",
        "content-format": 50,
        "if": ["core.ll", "core.b"]
        "href": "value",
        "rt": "urn:example:value",
        "content-format": 50,
        "if": "core.rp"
         "href": "ramptime",
         "rt": "urn:example:ramptime",
         "content-format": 50,
         "if": "core.rp"
         "href": "moveto/",
         "rel": ["alternate", "action"],
         "rt": "urn:example:moveto",
         "method": "create",
         "accept-format": 50,
         "if": "core.b"

GET /example/light/brightness/value?if=core.rp

         "n": "value",
         "u": "%",
         "vmin": 0,
         "vmax": 100,
         "vres": 0.1

GET /example/light/brightness/ramptime?if=core.rp

         "n": "ramptime",
         "u": "s",
         "vmin": 0,
         "vmax": 100,
         "vres": 0.1

The client knows how to invoke the action at this point:

POST /example/light/brightness/moveto/?if=core.b

         "n": "value",
         "v": 77
         "n": "ramptime",
         "v": 10

location: "77f3ac66"

The client knows that the location can be monitored for changes in the status of the action instance that was created.

GET /example/light/moveto/77f3ac66

         "n": "value",
         "v": 77
         "n": "ramptime",
         "v": 10
         "n": "status",
         "vs": "pending"
         "n": "remainingtime",
         "v": 10

Link bindings used to monitor resource state using REST callback

Perhaps the simplest case of a link binding involves monitoring the state of a resource and sending changes to a client using PUT to a REST API endpoint. This is often called a "web callback" pattern.

The resource being monitored is the source of updates and is the context of the "monitor" link binding, and the target is the destination resource, in this example the callback URI.

The link binding would look like this:

        "anchor": "/example/temperature/value",
        "rel": "monitor",
        "href": ""

Whenever the local resource at /example/temperature/value is updated, the representation of the resource will be pushed to the URI

By default, the source would use Observe or a REST hook and the destination (transfer method) would use update, by default PUT.

Using pubsub communication with REST resources

Link bindings can specify the source and target to be differnet protocols. For example, changes to the temperature value could be published to an MQTT broker, on an arbitrary topic name.

        "anchor": "/example/temperature/value",
        "rel": "monitor",
        "href": "mqtt://",
        "pmin": 10,
        "pmax": 600,
        "st": 1

The broker could be a reachable service in the cloud, and the temperature sensor could be in a sleepy device behind a NAT firewall, or in a mobile phone.

In this example, the conditional notification parameters indicate that the representation of the resource will be published when the value changes by 1 unit or more from the most recent publication, will be published no more frequently than once every 10 seconds no matter what the change, and will be published at least once every 600 seconds even if no change occurs.

Device-to-device orchestration using link Bindings

Link bindings may be used to orchestrate asynchronous updates from one device to another.

For example, an on-off switch could be push updates to a controlled device upon changes of state.

        "anchor": "/switch/onoff/value",
        "rel": "monitor",
        "href": "coap://[fdfd::9]/light/onoff/value"


Roy Fielding's Reference Work on REST

Roy Fielding's note on Hypertext and REST APIs

IRTF Thing to Thing Research Group (T2TRG)


CoRE Link-Format RFC 6690

CoRE Links in JSON and CBOR


CoRE Dynlink

CoRE Interfaces

Event Models for RESTful APIs

REST to MQTT Bridge slides

Sunday, May 15, 2016

Hypermedia Design for Machine Interfaces Part 2

Hypermedia Design for Machine Interfaces Part 2

Introduction and Background

The last article [1] described a hypermedia architecture and content format based on
  • Collections of resource items and hyperlink references to resource items
  • IETF CoRE link-format based hypermedia controls
  • Composite data representation using SenML

Since then, we have built a reference implementation of these ideas in the Machine Hypermedia Toolkit [2]. The primary goals were to create a reference implementation of the proposed content format and architecture from which to evaluate and further develop the core concepts including:
  • Hypermedia controls using RFC6690 and link extensions
  • Abstract transfer protocol mappable to HTTP, CoAP
  • Collection based resource model including links and data items
  • Content format mapping to REST interaction models
  • Link embedding and dynamic resource composition

The prototype implemented the proposed resource model, as well as a test framework consisting of rudimentary JSON-LD models, functions to construct resources from abstract semantic models, discovery of resources using resource model templates with semantic annotation, and abstract API binding to discovered resources.

In addition, a number of related research topics were explored and reports made to IRTF and W3C research groups [3]. An online demonstrator for the Machine Hypermedia Toolkit can be found at [4].

This was a very useful exercise and a number of valuable lessons were learned:
  • The structure of the base data model should be decoupled from the structures of the representation formats
  • A simple base model format should be maintained with a common namespace for link-params, link extensions, and data properties across different serializations
  • Asynchronous behavior should be designed into the request/response architecture using a consistent RESThook framework
  • Resource exposing scripts and callback handlers should have the same scripting interface as resource consuming scripts
  • A consistent model based interface for resource construction and discovery is useful and practical
  • It is necessary to expose some transfer layer information in order to bind interaction models to REST APIs; this is analogous to the idea of data in motion vs. data at rest

What next?

At this point the concepts are well developed and may be built out into a broader experimental framework. This is a summary of items for future development.

Define a content format appropriate for general machine hypermedia representation.

Some of the ideas need to be developed further into a concrete recommendation to enable broader exploration. These will be developed into a experimental draft for the IRTF Thing to Thing Research Group [5] (T2TRG).
  • Define a collection based data model
  • Define mappings to RFC 6690 [6] and SenML [7] representations
  • URI Processing rules, including query parameter processing
  • Define link embedding operations that enable dynamic composition of resource representations
  • Define a common set of hypermedia keywords based on current practices

Define abstract transfer layer operations that enable late binding to concrete transfer protocols like HTTP, CoAP, and MQTT

  • Enable resource handlers implementations of interaction models to understand data in motion in a protocol-agnostic way
  • Enable choice of protocol to be late binding based on content of a retrieved hyperlink

Develop a reference implementation of an abstract transfer layer and resource layer based on the specified content format and architecture [9].

  • Abstract transfer layer provides a protocol-neutral interface for APIs and device interfaces, Storage, and protocol bindings
  • Implement the W3C WoT Servient pattern using the transfer layer API for constructing and handling requests to exposed resources
  • Use the common resource and transfer layers to expose a scripting API
  • Scripts can construct resources to expose and discover resources to consume
  • Utilize W3C WoT Thing Description (TD) for resource models used in resource instance construction and discovery

W3C Web of Things

One goal is to create a reference implementation of the W3C Web of Things [8] architecture and the proposed JSON-LD based Thing Description language. The W3C WoT Architecture includes the concept of a Servient, which is a common interface for both consuming and exposing resources under control of an application or scripts. However, the W3C WoT group will not define specific transfer layers or resource layers.

Scripts that expose and consume resources, execute the "business logic" of things
Thing Description, Stateful Resources
Ethernet, WiFi, 6LoWPAN, Thread

Thing Description resources use URIs and other transport specific identifiers to point to concrete resource instances. There is a need to define resource layers and mappings to transfer layers for WoT TD models to encapsulate. The W3C Servient pattern will be the architectural use case for the abstract transfer layer and common resource model. The resource model can be configured to expose a simple REST + Notifications  API to expose WoT Event, Action, and Property resources with URI links.

Application scripts will construct and expose resources using Thing Description templates, with local callback handlers for resource access operations. The abstract transfer messages will be exposed to resource handlers in order to match hypermedia controls to the underlying interaction model.

Asynchronous Interaction Model

A simple RESThook based architecture can be used for subscribing to and receiving events from asynchronous data sources, including changes of measured properties and event sources. Using transfer layer patterns like CoAP Observe, MQTT Subscribe, and HTTP EventSource, the existing server transfer layer of request/response can be used to observe a resource for updates from other sources.

With this model, all interested clients register an OBSERVE or SUBSCRIBE request with the server. A hardware sensor driver or data source  simply uses the UPDATE operation on the resource, and all observing clients will be sent a notification based on their registered requests.

The problem with this model is that there is a hidden element of client session state on the server that can't be discovered externally. While this can sometimes be thought of as a request in progress, there is some motivation to build a more explicit mechanism into the system.

Using a separate resource we call a link binding (mentioned in [1]), the Observation on a resource can be made explicit by the instance of the binding. Each binding results in one observer on the resource. The binding contains instructions for constructing a message that can update another resource or perform an action like publish a message to an MQTT broker. These may in turn be observed or handled by actuators, etc.

RESTful Actuation

For RESTful actuation, the actuation command will be mapped to the CREATE of a temporary resource in a collection of actuations. When the actuation is invoked, the handler will create a resource that will be periodically updated to reflect the progress of the running action. For brief actions, the only update my be to signal completion. For longer running actions, there may be incremental progress updates.

The client wishing to perform an action will construct a payload representing the desired action (perhaps based on instructions in a hypermedia form) and perform a CREATE operation on the actuation collection. The resource handler will return the location URI of the created resource, which the client may choose to observe and receive asynchronous notifications of the progress.


[1] Hypermedia Design for Machine Interfaces Part 1

[2] Machine Hypermedia Toolkit project

[3] Research Summaries from the Machine Hypermedia Toolkit project

[4] Online Demonstrator for Machine Hypermedia Toolkit

[5] T2TRG

[6] RFC 6690

[7] SenML

[8] W3C WoT Interest Group

[9] Working documents