Sunday, February 22, 2015

Best Practices, Standards, and Tools for the Internet of Things - Toward an Ecosystem of Interoperable Devices and Services

Best Practices, Standards, and Tools for the Internet of Things

As Internet of Things developers, we can now build upon existing best practices, tools, and standards to create the next level of reusable and interoperable systems. We are on the verge of a new ecosystem of interoperable devices and services.

The previous article in this series discussed a number of design patterns used for the Internet of Things. IoT system architectures are constructed from these and other basic design patterns. Basically, architecture is a structured collection of design patterns that together embody a particular set of ideas and concepts to achieve a set of goals.

A set of best practices and common design patterns are emerging, as are the Tools and Standards to embody these best practices

Since that article was written nearly nine months ago, there has been substantial progress in the Internet of Things. A number of companies have fielded and test marketed a wide range of connected products and platforms. At the same time, we are seeing the emergence of best practices and common design patterns, as well as the steady evolution of a common set of open standards and tools that embody these design patterns.

This article will introduce an emerging set of design patterns, tools, and open standards which are analogous to those which provide the framework of the World Wide Web, but developed specifically for the Internet of Things.

Web Scale Reuse and Interoperability for IoT

One benefit of using common design patterns and tools is reusability. Once a piece of software is written or hardware designed to solve a particular problem, that design and often the implementation can be reused over and over again to solve similar problems in new applications.

Reusable hardware and software reduces the cost of developing new products and systems over time. One reason the World Wide Web has scaled so well is the reuse of software, and the resulting low barrier to new applications and services.

This is achieved on the web through a common set of information abstractions, or data models. These well-known abstractions and data models enable plug and play software interoperability between many different web services and web browsers.

But IoT is not the web...

For the web to scale, each information source must have a definitive origin, web servers where the state of the resources are determined, from which representations are derived and passed to consumers of information, the client applications (web browsers).

However, instead of connecting browsers to servers, the Internet of Things connects things to software and services. This is similar to the web in some ways, but very different in others. For example, the web server is the origin of information in the web of information, but the IoT Device, or thing, is the origin of information in the IoT. Whether the end device is a sensor or actuator, it is where the state of the system is ultimately determined.

This often inverts the traditional web mapping of clients and servers, making the smaller embedded computer in the device into the server, and the larger computer in the gateway or cloud into the client. It is necessary in this way for scalability, to allow the many-to-many relationship between things and software components to emerge, resulting in web scale IoT.

Data Models are Key to Building the IoT at Web Scale

Thus for the Internet of Things, the key to achieving the scale and interoperability of the web is in a common set of abstractions between things themselves, and the software and services that provide a layer of distributed intelligence in the system.

Figure 1 - IoT Application Interoperability Through Software Abstractions

Toward an ecosystem of interoperable devices and services

In the long term, the ability for many different applications to connect to and interact with many different sensors, actuators, and controls, that is the application interoperability, will become a huge enabler for new products and services. As the ecosystem of interoperable devices and services expands, so will the value of building new devices and applications within this ecosystem.

This emerging ecosystem isn’t owned by any one business or entity, but rather it exists to enable many entities to pool their resources together to create larger opportunities for all. Open interoperability delivers on the promise of open source software, enabling all boats to rise with the tide together, without interfering with anyone’s ability to compete by delivering a superior product and experience.

Open Standards are being adopted

Over last 12-18 months, we have seen the development and gradual adoption of some new open standards that will help to create a foundation for a reusable, interoperable Internet of Things. The rest of this article will describe these standards, how they work together, and how tools are available now to make it easy for developers to use these new standards and design patterns.

Based on the Internet Protocol

The proliferation of Internet Protocol (IP) and related design patterns has created a common point for the integration of protocols for a wide range of connected devices and services. Integration of IP into Bluetooth Smart has recently been developed, and likewise there is an effort to standardize Zigbee over IP. 6LowPAN and Thread are emerging with standardization and support in communication stacks.

Layered, Modular Standards

Rather than a single organization designing these protocols as an end-to-end monolithic standard from the radio to the application, the emerging standards are layered, allowing for the modular assembly of end-to-end protocols from the  technology appropriate for the use case at each level in the stack.

This allows for existing transport and network connectivity layers to be used when practical, and the newer low power radio technology and constrained device networks to be deployed where needed, all using the same application protocols.

Open standards, royalty-free

These emerging standards are openly available and usable on a royalty free basis, without any requirement to sign NDA or license agreements. It is a common requirement for companies participating in the development of these standards agree to disclose any related IP they hold which would be required to implement the standard.

A baseline example of this is the IETF Best Current Practices memos BCP 78 and BCP 79, which document the IP rights expected to be retained by the IETF and by participating companies as an outcome of the IETF standards process. The other standards bodies mentioned in this article either, have IP policies which are modeled after the IETF.

Practical standards, available now

These are practical standards have been published and are available now, with both Open Source reference implementations and commercial products available to end product developers and service providers. Please see the end of this article for specific references and links to get started.

Modular Stack

Figure 2 shows how a set of emerging Open Standards work together in a modular way to provide uniform Protocol, API, and Data Model components that enable end to end interoperability between constrained devices and web applications.

Figure 2 - Modular, IP based Communication and Information Standards for IoT

The bottom 3 levels are shown to illustrate how various end point hardware and networks can interoperate under a layer of IP connectivity.  This common IP protocol layer in turn provides a foundation upon which a common set of protocols and data models are based.

IPSO Smart Objects are extensible application oriented web objects based on the LWM2M data model and web object API. Basic sensors, actuators, and controls such as temperature, barometer, presence, etc. are represented using a simple common data model and resource template.

These basic objects can in turn be used as building blocks for more complex objects and machines like home appliances, e.g. a coffee maker that has temperature sensors, water sensors, heaters, user controls, timers, and displays. Figure 3 is the IPSO Presence Sensor Smart Object definition.
Figure 3 - Example IPSO Smart Object for a Presence Sensor

OMA Lightweight Machine To Machine (LWM2M) is a profile for IETF CoAP and related standards. LWM2M specifies a set of common interfaces and data models to enable plug and play interoperability between CoAP devices and local or remote services.

LWM2M provides necessary device management functions like secure device on-boarding and Over The Air (OTA) upgrade and configuration of devices, using a standard set of device management objects.

LWM2M defines a simple object model using the CoAP REST API, a device management architecture using REST objects, and system features including control of asynchronous notifications. This simple object model is the basis for IPSO Smart Objects.

Figure 4 - OMA Lightweight M2M (LWM2M) Server Architecture

IETF CoAP (RFC7252) is a simple and compact wire protocol and binary mapping of the REST design pattern, optimized and extended for efficient machine to machine (M2M) communication in constrained resource environments. Asynchronous notifications, publish-subscribe, and resource discovery are standardized by the IETF for use with CoAP.

CoAP devices can operate in both client and server roles, enabling device to device interactions as well as interactions with application and management middleware. Figure 5 shows the Server role of a CoAP temperature sensor, where the application software has a client role in requesting the temperature from the sensor.

CoAP also provides a built-in protocol for the sensor to send a continuous series of asynchronous state updates to the client application, similar to web-based server push messages.
Figure 5 - CoAP Device as Server

Related IETF standards for web linking of constrained devices provide a framework for semantic discovery of resources by peer devices and applications. Resource Directory is a service layer that enables collections of resource descriptions, in the form of web links, to be cached in a server for applications to use in resource discovery. LWM2M uses Resource Directory as its device registration interface.

Standards Work Together for Modular End to End Interoperability

As with the web, a good way to achieve end to end interoperability between devices, services, and applications, is to use a common set of abstractions and data model. Following best practices from the web, this data model should be relatively independent of transport protocol, and focus on the information and semantics which are relevant to the application. IPSO Smart Objects provide a transport-independent common data model, which is simple and extensible, on which to start building end-to-end interoperable systems.

At the same time, the standardization of CoAP and LWM2M enable a plug-and-play ecosystem of IP-connected devices that can connect using different physical networks like WiFi, Bluetooth Smart, Thread, and even wired ethernet, all to the same services and applications.

LWM2M management objects and IPSO Smart Objects enable application software and device management software to use the same web object API to interact with devices. Existing LWM2M server software will support both LWM2M management objects and application objects (IPSO Smart Objects).  

The CoAP protocol allows for device-to-device local network interactions, which can in turn re-use LWM2M interfaces and management objects as well as IPSO Smart Objects.

IPSO Smart Objects can be used over HTTP, which enables the LWM2M server to act as a reverse proxy between constrained network devices and standard web applications. HTTP can be used between constrained device gateways and local apps on devices like smart phones and tablets.

IPSO Smart Objects can also be used with other protocols, for example using MQTT for asynchronous updates of object and resource state between gateways and cloud services, and for cloud-to-cloud data connections.

Larger Ecosystem is Converging Around These Standards

As the above-mentioned standards are being developed, there are other standards organizations incorporating them into their own standards development. Rather than develop new standards, these organizations are choosing to incorporate and integrate existing modular standards. This gives them a wider range of applicable use cases, since they can choose more optimum wire protocols, for example using CoAP over SMS.

OneM2M is a global consortium of network operators, service providers and related enterprises that are creating a very broad set of standards to address M2M and IoT use cases. OneM2M includes interworking with CoAP and LWM2M to provide interoperability between devices and M2M services. This makes IPSO Smart Object devices interoperable on OneM2M platforms which support these interworkings.

Likewise, the Open Interconnect Consortium will integrate existing standards, and have adopted CoAP as a constrained device wire protocol.

“The Open Interconnect Consortium (OIC) will seek to define a common communication framework based on industry standard technologies to wirelessly connect and intelligently manage the flow of information among devices, regardless of form factor, operating system or service provider.”

Tools are available now from multiple sources, enabling an ecosystem of reusable software based on open source and open standards

IoT developers have access to these design patterns and standards through an ever-expanding set of both Open Source and Commercially Licensed tools.

Since the protocols are all IP based, the tools are mainly focused on the machine-to-machine endpoints of the system. One endpoint is in the devices with embedded software. Tools for this endpoint consist of embedded IDEs and operating systems, covering the range from microcontrollers like ARM mbed and Arduino, to small microprocessors like the Raspberry PI running embedded Linux environments (DD-WRT, Yocto, Tizen).

The other endpoint is an application service layer, or middleware, that devices connect to and through which applications can discover and interact with connected devices. LWM2M middleware provides a persistent endpoint through which devices and applications can interact in a scalable and secure way. Figure 6 shows resource discovery, REST API operation and asynchronous notification (push updates) using LWM2M.

Figure 6 - LWM2M Application Proxy Operations

Open Source developer tools available for LWM2M Devices :

ARM mbed - Open Source web based IDE for Cortex-M microcontrollers with a LWM2M device client in C++, also a Java LWM2M Client is available for larger devices.

Eclipse Wakaama - is a linkable code base for embedded endpoints, implements C++ LWM2M device client and server endpoints. Can be used with ARM mbed, Contiki, and other embedded system tools.

OMA LWM2M DevKit - Developer tools from the Open Mobile Ailiance for building and simulating LWM2M endpoints

For LWM2M Middleware and servers, there are both commercial license and open source licensed products available:

ARM mbed Device Server - Commercially licensed server - free developer download:

Eclipse Leshan Open Source LWM2M Server tools - Java implementation of LWM2M

OMA LWM2M DevKit (support for server in progress)

Standards and tools mentioned in this article

IPSO Smart Objects





OneM2M - has Interworking with CoAP and OMA LWM2M

Open Interconnect Consortium (OIC) - uses CoAP as wire protocol, developing use case and application profiles

ARM mbed IDE

ARM mbed Device Server

Eclipse Wakaama

Eclipse Leshan

OMA LWM2M DevKit and LabKits

Monday, May 26, 2014

Design Patterns for an Internet Of Things

Design Patterns for an Internet Of Things
A Design Pattern Framework for IoT Architecture

Design Patterns are reusable solutions to common problems

Design Patterns provide well known ways to solve design problems commonly encountered in a particular discipline or problem domain. As an example, three different design patterns to handle traffic flow at a road intersection are stop signs, traffic lights, and roundabout. Each have advantages and disadvantages specific to particular traffic patterns and other contextual factors.

IoT presents design problems in many areas and at many levels in the system. There are many diverse use cases, with different resource constraints, and with many standards, products, and technologies available. How do we determine which are appropriate, what the specific advantages and disadvantages are without a context from which to evaluate their use?

Design Patterns are building blocks of architecture

Design patterns provide a way to build an end to end solution in well specified ways and to provide an understanding of the use of different components of the system in a system context. A reference architecture can be constructed from a set of design patterns, and from this the behavior of the system may be modeled and understood.

The articles published here up to this point have focused on a set of design patterns around data models and information models for the Internet of Things, and most recently began investigating an open stack approach, where use-case appropriate end to end solutions can be built from various components and design patterns.

Next I would like to look at a set of system design patterns useful in the construction of IoT architecture solutions, with a focus on common patterns for interoperability. I have divided the design patterns somewhat arbitrarily into areas of information models, interaction models, application programming models, infrastructure models, and use case patterns.

The design pattern examples were drawn from www and internet, best practices and standards, those which appear to be successful and well known, and those which satisfy some need or solve some problem. These are only examples and not meant to be exhaustive. However, the intention is to enable covering most common IoT use cases using a set of design patterns which can use  well known standards and practices.

Each of these areas is further decomposed into a spectrum of higher level patterns built on lower level patterns. I present only a few examples of common design patterns; this is not meant to be an exhaustive treatment of the subject.

Design patterns for connected things represents the fundamental proposition of an Internet of Things; that is connecting things using networks and software. Here are some examples of fundamental design patterns for IoT:

  • Software connected to thing via a network: the fundamental proposition of the IoT is that connected software is of higher value than embedded software. Connected software is less resource constrained and can integrate more diverse data sources.

  • Virtualization: Software connecting to an abstract representation of the thing. Virtualization makes it easier to create reusable software and devices.

  • Virtualization through middleware: allows many (web) applications to interact with things. Middleware can cache the state of the thing and minimize network traffic and power drain on constrained devices, and can also serve as a persistent end point for things that aren’t reachable over the network due to power cycling, firewalls, etc.

  • Middleware Platform: Allows many-to-many interactions between applications and things. Enables realization of connected environments and network effects. Can provide standard interfaces for things and application software.
  • Thing-thing interaction: things that contain some application software interact directly with other things on local networks

Design patterns for IoT use cases describe the system level use case mapped onto high level design patterns like gateways and web services:

  • Devices talk to other devices peer-to-peer: local network connectivity enables proximal ad-hoc networking, service federation and chaining, media stream continuity.
  • Personal tracking device uses smartphone as gateway: common pattern for bluetooth and WiFi connectivity.
  • Smarthome local application controller and gateway: application gateway pattern.

  • Monitor a large number of devices over a large area: collect, filter, analyze pattern used in Smart City, surveillance systems, large scale resource monitoring.

  • People interacting with autonomic feedback loops: general purpose use case involving people and automation in defined roles. Autonomic control relieves people of the job of monitoring, but lets them be involved in high level control and exception handling.

There are many more design patterns at all levels of design. The following list contains some more common patterns based on modern web patterns and practices that are relevant to IoT architecture.

Design patterns for information models consist of lower layers of data models and representation, upon which are built higher level encapsulation and function. Some examples of information model design patterns:
  • Structured data: XML documents, JSON objects
  • Web Objects: multiple resources at a URI endpoint, object encapsulation
  • VIrtual Objects, Smart Objects: a set of resources that represent a physical thing or other data source
  • Composite Objects: Virtual Objects composed of resources from other objects
  • Hypermedia, HATEOAS: Applications use pointers to URI endpoints that externalize application state
  • Semantic Hyperlinks: Hyperlinks with embedded semantic tags, e.g. relation=value
  • Information model: Collection of semantic hyperlinks describing a resource
  • Context model: Information model layer describing context of a set of resources
  • Binding model: Information model describing dynamic binding of actions to resources
  • Resource Directory, Catalog: A collection of model instances describing sets of resources
  • Resource constructor: Information model that informs the construction of resource instance
  • Access control model: Information model that specifies access control policies and constraints for a set of resources

Design Patterns for Interaction describe how different parts of a system interact and communicate with each other, including communication protocols. Some examples:
  • REST: REpresentational State Transfer, design pattern allowing for externalization of application state in reusable, shareable resources
  • Asynchronous Events: State updates propagate through the system as they occur
  • Resource Binding: Associating a resource with an action, bridges REST to Asynchronous Events
  • Observer Pattern: A binding of resource updates to a protocol action or handler
  • Publish/Subscribe: A communication pattern where a client registers interest in a topic by subscribing, updates to a topic are published to all subscribers
  • Broker: A central service to connect publishers with subscribers
  • Proxy: A machine that provides an interface on behalf of another interface
  • Protocol Bridge: A bidirectional translator between two protocols
  • Resource Discovery: A process where resources are found by specifying attributes
  • Resource Registration: An endpoint informs a resource directory of it’s resources
  • Sleeping/Non-reachable Endpoint: An endpoint is not reachable and must participate in protocol by initiating all interactions with reachable or always-on endpoints

Design patterns for Application Programming describe ways that software and interfaces are created, managed, deployed, and used in IoT applications.
  • REST Objects: Mapping of REST API resources onto program objects in the application language, using libraries
  • Event handler, onEvent: application code that responds to asynchronous events
  • Event driven flow: a set of application handlers that operate in an event driven graph containing series cascade and parallel constructs
  • State Machine: a logic construct where a next state depends on a set of inputs and the current state, evaluated by a set of logic rules associated with each state
  • State Externalization: the ability to create stateless application software by mapping application state onto external resources
  • Rule oriented programming: using a set of rules or rule language to program state machine logic
  • Abstraction of applications: stateless application software uses application templates for reusability
  • Application templates: abstract application components with well defined interfaces
  • Modular applications: applications consisting of one or more reusable components
  • Applications run anywhere, location independent applications: Application components can run anywhere, in devices, on local network servers, in gateways, in edge servers, in cloud, on user devices.
  • Discovery and Linking: Integrates resources into applications by resolving resource links, sets attributes in application objects
  • Object Constructor: Creates application software objects from metadata models

Design patterns for infrastructure describe how different network and device technology is used to solve problems with the physical infrastructure of IoT. How do low power devices connect to wireless sensor networks and ultimately connect to services and applications:
  • 6LowPAN edge router: moves packets from 6LowPAN network to IPV6, does header compression
  • WSN access point: mediated access from WSN network to IP backbone using a joining protocol e.g. WiFi
  • Mesh routing: routing network protocol messages through other endpoint nodes in a network
  • Application gateway: device with both network connectivity and the ability to run application components locally
  • Behind-NAT connectivity: using reachable service or broker, applications and devices connect to each other from behind NAT firewalls
  • M2M WAN: Wireless Service Providers specialize in M2M and provide wireless WAN networks, e.g. SigFox

Design patterns for IoT security describe design patterns for IoT security problems.
  • Access control using data models: semantic hyperlinks control access to resources based on the embedded metadata
  • Social to physical graph relationship: well defined concepts of ownership and access delegation between people, entities, and things
  • PGP and asymmetric public-key cryptography on devices: ways of creating SSL sessions and signing data between devices and applications
  • DTLS over UDP: security for resource constrained devices
  • End-to-end encryption: transmitting and storing encrypted data independent of channel encryption                                                                                                         
  • Device Management: using device identity, registration, and secure key exchange

There is no one “reference” architecture for an Internet of Things. Design Patterns are a way to construct architecture solutions for specific use cases and use case classes.

The work to find system architecture solutions to Internet of Things problems has led to an obvious conclusion that there is no single architecture appropriate for most IoT use cases. The full spectrum of IoT presents a broad range of diverse use cases and resource constraints, and thus motivates a range of architecture solutions. Still, we would like to ground the discussion in a reference set of technical concepts, to help promote a unified understanding and break down the silos of thought around IoT architecture. We would also like to find opportunities for standardization and commonality.

Different architecture solutions are appropriate for different use case classes, and architecture is expected to be reusable within a particular class of use cases.Therefore it makes more sense to talk about IoT architecture as a set of Design Patterns, working together to achieve an end-to-end solution for some problem.