Sunday, December 30, 2012

The Internet of Things needs an Open Infrastructure

The Internet of Things needs an Open Infrastructure

Michael J Koster

An Internet of Things application may be best represented by a graph, with sensors and data sources connected to filters, triggers, rule evaluators, and other processes, with these outputs connected to actuators, storage, visualizations, etc. The connections are made according to user intentions and are customized to the user's context.

It's often observed that one thing holding the Internet of Things back is the difficulty of connecting things together. Some are trying to solve this at the platform level, by providing connectivity within the platform e.g. SmartThings, and some are working to connect data between vendor's offerings, e.g. IFTTT. 

A general solution to the problem would be a common connection and communication mechanism between the various sensors, actuators, gateways, services, and apps. It's generally realized that some degree of "open-ness" is needed for the systems to be connected.  Various claims of open-ness are being made for different IoT systems and in different contexts. In this post I would like to discuss the meaning and impact of open participation in various aspects of Internet of Things technology and processes.

First, let me describe the meaning of some common concepts as I understand them for the purpose of this discussion:

What is a Platform?

A platform is a common set of resources used by developers to build and deploy applications. There are other types of platforms, but for this discussion there is a common set of resources used by IoT developers on which to build IoT applications. 

For the common IoT deployment model of sensors, data sources, and actuators connected to services, the platform could consist of sensor nets, gateways, service enablement, APIs, servers, clients, application libraries, smartphone UX enablement, and other common resources. The platform would allow application developers to build or add sensors as needed and develop applications that can be hosted on the sensors, gateways, services, and smartphones. A minimum of customization of the platform should be necessary to support new applications. 

Platforms allow developers to share resources and patterns, thus minimizing the new code needed to create new applications. Platforms are often specific to application domains; there may be many different platforms across the IoT just as there are many platforms on the WWW. A platform is often defined by a programming language or an operating system, e.g. "the Windows platform" or "the Java Platform". Applications built on a particular platform are often interoperable with other applications on the same platform, and portable from one instance or implementation of the platform to another.

Examples of platforms that are built on the WWW are WS, .NET, AJAX

We could say that the WWW is itself an application built on IP, where IP is a platform. Other IP applications include ftp, smtp, POP, IMAP. 

What is Infrastructure?

Infrastructure consists of common resources used for deployment of platforms. 

Physical infrastructure includes the copper, fiber, silicon, and wireless switching fabric of the internet.

DNS is an infrastructure service for the Internet. 

Anytime some layer in the stack becomes ubiquitous and is used to build platforms on, it could be considered infrastructure. IP could be considered infrastructure on which the WWW runs.

The IoT infrastructure consists of wireless networks, the Internet, routers and switches, and in many ways the basic protocols used. For example, HTTP and CoAP enable IoT platforms to be built using these web protocols as APIs to the common internet infrastructure.

What is an Ecosystem?

An ecosystem is a system of systems that all share resources and protocols, and interact and influence the outcome of each other. The WWW can be considered an ecosystem. The Internet and supporting protocols HTTP, etc. are it's infrastructure. There are a number of platforms that all interoperate, more or less, within the WWW as an ecosystem. 

In a healthy ecosystem, it's natural for platforms and protocols to come and go. There is a natural interdependence that is balanced against competition. The concept of "all boats must rise" is important to an ecosystem. 

In this context, it's difficult for an ecosystem which has multiple competing platforms to be contained within a single company or institution. The sustainable ecosystem seems to be based on open participation at all levels.

What is Open?

There may be many definitions of "open", but for this purpose it should refer to an open avenue of participation. By this I mean that any capable participant is allowed to participate on an equitable basis. Participation can occur at many levels in an Internet based system; here are a few example categories:

Open participation for Users

Participants at this level trade for goods and services that benefit them in their life or business. Open participation for users has to do with access to goods and services at fair prices with fair policies, through accessible channels and outlets, and enabling user participation across languages, etc. without any obligation or limitation on the part of the user as to their fair use of the goods or services, including sharing what they own and control with others.

Internet of Things users will participate in creating their own applications and managing their own information in ways not previously possible. The role of user as composer of IoT application graphs gives the user agency in customizing their own experience. At the same time, it requires a new level of compose-ability in services and data sources. It will also drive new solutions to today's problems of password based security and user control of information. 

Open platform for Developers

Participants at this level add value to goods and services by creating new offerings based customizing, programming, combining, or otherwise building on top of the offering as a platform. Open participation here requires equitable access to specifications and design information without requiring intellectual property licenses, regardless of institutional affiliation or lack thereof. It should be possible for a developer to create a new application on an open platform without encountering barriers to entry. 

Internet of Things application developers will need to provide high level compose-able services and data sources, in a way that users can easily manage. There will need to be standard, frictionless, transparent mechanisms to pass user authentication and provenance along with commands and data, built into platforms. This will require a level of cooperation between developers and standards bodies throughout the process.

Providers (Service Provider, Hardware Manufacturer)

Open access for service providers enables multiple sources. Open participation at this level requires open access to specifications and access to the standards process, without any licensing restrictions, and regardless of institutional affiliation or lack thereof. It should be possible for a start-up company to enter the business as an open platform service provider or manufacturer without barriers.

Standard infrastructure services, APIs, and platforms will enable a service provider business model, where someone with domain expertise and an idea to solve a problem can easily build a new service business focused on solving the domain problem. Open source reference implementations of infrastructure components will enable IaaS and PaaS providers to support rapid prototype deployment of new platforms and services.

Standards Body

Standards bodies are responsible for the common specifications of a platform or system of platforms. They are usually composed of representatives from corporations and public institutions, and experts in the field. An open standards process is one which allows equitable participation of any qualified person, without any licensing restrictions, regardless of institutional affiliation or lack thereof.

There are currently a number of different standards processes being used to develop IoT standards, each resulting in subtly different qualities of open participation at the different levels. Here are a few examples of broad categories.

Types of Standards

Examples of standards process include de facto, Industry SIG, and the Internet standards process (IETF). 

An example of an industry de facto standard is Microsoft Windows. In this case, avenues for participation are limited to a single vendor provider and single corporation develops the specification, i.e. only one vendor provides the Windows platform. Participation by software application programmers is most open, with practically no barriers. User participation is limited by a restrictive end user license. 

Many current standards are based on Industry Special Interest Groups (SIG). A common SIG purpose is to share and cross-license Intellectual Property between corporate and institutional members. The specification is often not publicly available, but can often be obtained by joining the SIG and executing the license and NDA. An example of this type of SIG is the Weightless TV whitespace M2M standard. Participation as a developer or vendor is limited to SIG members. Participation as a user is usually based on a hardware purchase or in the case of software a binary license.

The Internet standards, for example the Internet Engineering Task Force (IETF) are based on open participation and industry consensus. Avenues for participation are mostly open to all, at all levels. Tim Berners-Lee didn't have to ask permission from a TCP/IP SIG to develop the WWW application and HTTP.

In Europe, there is a broad consortium of Industry, government, and educational institutions working with standards bodies to develop a comprehensive framework for the IoT. This is an open process that allows many avenues of participation. There isn't enough space here to break it down, but there are some excellent participatory models being used. 

What's the license?

When it comes to answering the question "How open is it?", the final word is the license. Any formal work these days, whether software, hardware, or process, e.g. a standard or benchmark, comes with a license. It's the details and requirements of the license that ultimately determine how open your avenues of participation are. It's worth noting that the term "open" is used by different providers to describe a wide range of license restrictions, and it's very important to understand exactly what the license requires and how it limits or enhances your value in participating.

Examples of licenses include the Microsoft Windows EULA, the GNU General Public License (GPL), THe Apache 2.0 License, The Facebook Terms of Service agreement, and basically every long document you need to scroll through and check "agree" on before participating in a service or downloading software or activating a hardware.

Open Source software

Open Source software is generally recognized as software which is available in human-readable and maintainable source code form, to any developer that follow the provisions of the license. Open Source software has both copyright and license. The concept of "copyleft" is sometimes used to describe open source license terms. Some open source licences require derivative work to be published with the license intact, e.g. the GPL. Other licenses do not restrict the use of the code, and allow a developer to produce closed source derivative works. 

Sometimes source code is published for informational purposes, for example to write interface code to an API, but is not available under an open source license, that is the work may not be reproduced or used for derivatives. This is not open source.

It's generally agreed that infrastructure services and platforms benefit from being Open Source, where multiple contributors are involved in revisions and bug fixes. This provides a level of transparency that has been proven to result in very robust code.

The infrastructure of the Internet, that is the DNS servers, web servers, programming languages, and client components, is built from Open Source software. SInce the Internet is an ecosystem, or system of systems, there are compatible closed license source alternatives for many of the services. 

Open Source Hardware

Lately the notion of Open Source Hardware has become popular. The basic idea is the same as Open Source software, in that the design specifications, schematics, board layout, and firmware are available for anyone "skilled in the art" to reproduce the hardware for any purpose as long as the terms of the license are followed. A good example of this is the Arduino. 

Again, it's the license that matters, even if the design specifications are published for reference. Arduino is available under an open hardware license that allows anyone to manufacture and sell the designs. Note that this is not the case with Raspberry Pi. The Pi hardware license is controlled by the Pi foundation, and is only made available on a case-by-case basis to their manufacturing partners.

Open API

An Application Programming Interface allows developers to build applications on a platform or to communicate with other applications and services. The idea of an open API is to open the system at a particular point to allow developers to build onto the system without signing a license or NDA. This gives the platform provider control over how developers can use the platform, while allowing anyone to develop on, and add value to, the platform. 

The API a platform provides becomes essentially an extension to the application programming language, baking the API into application programs developed on the platform. The unfortunate consequence of this is that applications developed on one platform and API can not be used, or sometimes even ported easily, to other platforms that use different a different API.

So very often, an open API is a tool for a platform provider to attract developers and lock them into the platform. Sometimes the lock-in is not intentionally done to exploit the developers, but rather an unintended consequence of simply developing a unique API for a new platform. That is, there is no well known API that exists, no a new one is created.

Open Source vs. Single Source

Another impact of closed development is the creation of a single source infrastructure. For example, a number of the current IoT system deployments include sensors, a gateway, and a cloud service. For the system to perform it's intended function, all components must be functional. If any component fails or if the provider of a component goes out of business, the entire system fails. Many of the vendors have addressed the sensor/actuator part of this equation by providing an open API or open source client code, etc. but still present a single point of failure in the cloud service. 

In this sense, then, the opposite of Open Source is Single Source, and the attendant single point of failure. Of course, we need to provide value in unique services that, if open sourced, would allow competitors to offer equivalent services... Where is the business model? 

One solution is an open source service enablement layer that allows the creation of a service ecosystem. Vendors can offer different, competing services that are targeted to user groups and use cases. Users can have a choice of differentiated services to perform a particular function in the application graph. The Internet of Things is going to be big enough to not only enable this, but soon to require it.

Impact on innovation cycle

When everything was developed under the closed IP based system of the industrial age, innovation proceeded at a pace regulated by the patent system and the pace of product development cycles. We had to wait to see what the competitive field produced in secret and then released in public to drive each innovation cycle. These cycles often got out of phase, resulting in some spectacular self-indulgent technology spirals like the iAPX432 or, more recently, the CPU pipeline MIPS blowout ca. 2000 where, out of competitive pressure, all the vendors produced designs that were way too expensive and couldn't be cooled. Upon further thought, it may be engineer turnover between companies that becomes the rate limiting factor in the closed IP development world...

On the other hand, the opening up of software engineering that came with Open Source and Open coding platforms like Python, PHP, etc, has resulted in sweeping innovation of the software process, with agile development cycles measured in days and continuous release processes. Use of common patterns like RESTful APIs and platform support for common patterns have resulted in most software effort focused on building an implementation of the idea, instead of building scaffolding and plumbing.

Access to one's own inventions

I hold a number of patents in computer architecture. Actually, I don't hold them, I assigned them to my employers who sold them to their buyers etc. and as a result are now owned by completely different people than when I made the inventions. I could expect to be sued by these people if I try to advance my work in these areas for another company, or at the very least my prior patents could be used against me. 

What I'm getting at is the current system of IP assignment and value based on secrecy is a limiting factor in my development of any technology I worked on while under the IP agreements imposed by the corporations that employed me. In reality there is a concept of fair use but no one knows exactly what it is without a legal test. All this creates a bias in me to want to work on open source systems where I can maintain access to my own work.

Impact on scalability and single point of failure

Open systems have a distinct advantage in the ability to rapidly scale up and to tolerate single points of failure. IN an open Source ecosystem, if one company fails or is unable or unwilling to continue maintaining a component, there will be another to take over or sometimes provide resources to keep the project going. In a closed system, the single points of supply become the weak links. 

Consider the Raspberry Pi. The single supplier situation combined with unexpected demand has resulted in continuous shortages, to the point of being a business risk for anyone considering using the Pi as a component. If the RPi design were licensed as Open Hardware, there may have been an opportunity for an second source supplier to come up to speed and meet the demand. At least the business risk would be mitigated by the possibility of an independent second source.

Business models

One item on every "must have" list for outside capital is something no one else has, preferably with a lock on the Intellectual Property. The requirement for exclusivity should be balanced against the advantages to everyone of a healthy sharing of infrastructure. 

It's often assumed that a business must possess some secret in order to be competitive. It's also assumed that Open Source programmers don't get paid. Both these notions should be challenged. Plenty of businesses thrive without needing to keep secrets. Businesses based on Intellectual Property are relatively new. It's worth noting that most of the modern infrastructure around software patents was created during the ascent of the Microsoft corporation, with much help from a certain Mr. Gates Sr. in the legal department.

The Open Internet of Things provides clear opportunities in two areas. The first is in providing new hardware for sensors, actuators, gateways, and data centers plus the required hardware connectivity in sensor nets and public infrastructure. The second is in providing services that add value based on the rich connectivity afforded by an open infrastructure. 

These devices and services can and should certainly be unique and don't need to be Open Source. The important thing is to not create lock-in or lock-out situations which result in unfair access or single points of failure. If there is good use of a common set of protocols and infrastructure, the ecosystem should support plenty of redundancy in heterogenous solutions and open source alternatives. 

I believe the best business models will be the ones that take advantage of connectivity and standards, rather than the ones that create vertical silos, walled gardens, and lock-in, regardless of the exclusivity of the Intellectual Property. Besides, recent legal battles illustrate the futility of pretending to own something everyone uses or does.

The Emerging Internet of Things needs an Open Infrastructure

To realize the Internet of Things application graph shown in the introduction, a level of open-ness is needed at all levels of participation. Open standards are needed to enable broad industry participation without any legal obligations. Open platforms are needed to allow users to compose, share, and migrate their application graphs across multiple providers. 

Open standards and platforms enable developers to focus on the business model, domain oriented, value added, areas of devices and services, and allow the cost of developing and maintaining shared infrastructure to be shared across the industry. This is facilitated by the adoption of Open Source platforms, both hardware and software, for the emerging Internet of Things infrastructure.

Friday, November 2, 2012

The Open Source Internet of Things

Friday, November 2, 2012
The Open Source Internet of Things

Smart Object API and Internet of Things Gateway
The Smart Object API and IoT Toolkit is a reference implementation of an Internet of Things gateway and application service. Earlier posts in this series describe the API and framework at a high level. To summarize, a Smart Object encapsulates the observable and descriptive properties of sensors or data sources into a RESTful web object (URI) with Linked Data compatibility.

I have been writing code and refining program API details of the reference implementation for the last few weeks. It's still very much an early work in progress; working toward an early collaborator review and functional demonstration around November 15th.

For those interested, the code is hosted at github:

NB I'm still working on some of the very basic functionality. This provides a look at the patterns and methods. There is a basic http service with some example objects and resources. The service is itself a Smart Object with it's high level Semantic Description, which registers and describes the set of Smart Object instances in the local gateway. Discovery and linkage will be layered onto the service object. The http server has a resource dictionary interface to the object service.

Remaining work items:
- Implement bug tracking
- API and unit test cases
- Threaded queue for network clients
- CoAP server and proxy
- Object building, cloning, and persistence using Semantic descriptions
- Semantic discovery and linkage, core-link-format compatibility
- Auth model (httpauth, oauth, WebID, API keys)

IoT Gateways and IoT Gateway-as-a-Service 

The goal is to enable a commonly used IoT deployment pattern:

sensor net <=> IoT gateway <=> Internet service <=> client interface

The target deployment is an Ubuntu image that can be deployed as:

- Installation on Raspberry Pi, supporting the Ciseco EVE IoT gateway with multiple onboard WSNs
- Gateway-as-a-Service e.g. Amazon EC2 micro instance with application running in cloud

The gateway and service work together with a common API, allowing applications to be distributed between multiple gateways and service instances. The gateway pattern will be able to integrate third party services and sensor networks.

Each gateway user or owner would use a dedicated cloud service instance or set of instances. Cloud services scale up by adding more instances (virtual machines) which isolates users and owners from each other and allows for granular redundancy and load balancing.

Client-only applications will be able to use the API as a library for interacting with services or directly with gateways.

The IoT Toolkit Open Source project
The Smart Object API and IoT Toolkit is a collaborative Open Source project. There is still a lot to be defined and built to enable end-to-end deployment, and another layer of application enablement is needed.

The IoT Toolkit will provide:
- RESTful API for data source, gateway, service, and software agent peer interaction model
- Application framework for gateway and granular scale cloud instances (GaaS)
- Semantic Web linked-date discovery and linkage, CoAP Resource Discovery compatibility
- Multi-protocol IoT gateway including CoAP-to-HTTP proxies and sensor net agents
- Serial asynchronous communication tools for low level sensor nets
- Ubuntu, Raspberry Pi, Amazon EC2 images
- Sensor and client reference implementations using open source tools e.g. Contiki

Different real world and experimental deployments will provide robust test cases for defining API features, as well as the next larger ecosystem of index and search, ontologies, client tools, etc.

We're launching a website, , to provide a point of contact for this project. The goal is to enable end-to-end deployment of IoT solutions through a set of Open Source tools and facilities  for Sensor Nets, Proxy Gateways, Gateway-as-a-Service instances, and Application software integration.

The Open Source Internet of Things 
The World Wide Web as we know it is made up of Open Source platforms and services. Apache, Mozilla, Ajax, Perl, PHP, Python, Linux, Android, MySQL, Hadoop, OpenFlow, the list goes on. There is a common set of tools and platforms that the community maintains as a shared infrastructure on which to build new services.

There is no such shared infrastructure yet for the developing Internet of Things. I believe there are many people working on creating it, some believing that there could be a common set of languages, tools, and platforms on which to build the next level.

In this spirit we would like to engage a broad discussion on the Open Source Internet of Things. We've created a new Open Source Internet of Things meetup group :

We would like the discussion to include engineers, entrepreneurs, and enthusiasts from anywhere, and to facilitate new projects, collaboration, and sharing. We believe that the next level is at hand, where we build a common platform for Internet of Things communication and applications that we can use to build domain specific services and applications on top of.

More Information

The Smart Object API is described at a high level here :

The Smart Object work-in-progress code is at:

The EVE board is an IoT gateway board from Ciseco with Xbee, EnOcean, SRF, Zwave, and RFM12B radio modules plus real time clock and onboard sensing:
EVE Hardware Kickstarter

Raspberry Pi is a $35 ARM based single board computer that runs Linux:

The Open Source Internet of Things Silicon Valley meetup

IoT Toolkit Open Source project:

Sunday, October 7, 2012

RESTful interfaces for Internet of Things gateways

RESTful interfaces for Internet of Things gateways

Previous posts in this series have introduced the idea of standard data models for the Internet of Things and proposed the Smart Object API as one such data model. The Smart Object API provides a RESTful web object encapsulation of self describing resources including Observable Properties, Software Agents, and publish-on-change Subscriptions, also known as Observers.

In the last post, I showed a channel model for the Internet of Things which consists of constrained networks connecting to the Internet through gateways.

In this post I'd like to explore the IoT gateway, it's architecture, and it's internal resources and operation.

Gateway model for the IoT

In the current IoT deployment, the channel model shown above is very popular. Typical IoT deployments consist of constrained sensor nets connected to cloud services through gateways.

I'd like to broaden the model out a bit, to include a number of, possibly heterogenous, constrained sensor nets connected to the internet, connected to each other through the internet, and connected to shared services through a common protocol using the internet.

In this model, the separate sensor net protocols are federated, using gateways, to a common protocol based on HTTP and RESTful interfaces. Common resource protocols and data models allow relatively easy sharing and federation of sensor data at a shared service. A single user app can be used to interact on all the different sensor networks.

Proxy Gateway patterns

The gateway operates on the local sensor network on behalf of a user device, service, or other sensor network, connected through the Internet or LAN.

The gateway operates on the Internet or LAN on behalf of devices on the sensor network.

Operation on behalf of another entity across a gateway can use some well-known patterns:

Forward Proxy
Proxy which is used by the client to access server resources across the gateway. Client "knows about" the proxy and interacts with proxy resources.

Reverse Proxy
Proxy which looks like a server to the client, i.e. the client interacts directly with the server resources across the gateway.

Interception Proxy
Proxy which transparently maps resources and protocols both directions, i.e. CoAP resources look like http resources from the http side and http resources look like CoAP resources from the CoAP side. This might also be called a transparent bridge.

A client-side proxy maps onto the client network, and a server-side proxy maps onto the server network.

A proxy may be caching, able to retain a copy of the return value of the last request which is used to satisfy subsequent requests for the same resource.

Gateway Capabilities

The gateway can add it's own resources as a proxy to provide additional capability to enable sensors to interact on the internet. For example, security and Contextual Semantic Web linkage can be "added" to the sensor by the gateway.

Another example is connecting CoAP sensor nets to the internet. CoAP has some useful capabilities, including semantic description of resources through the /.well_known/core interface (core-link-format) and data push capability. A gateway can provide HTTP emulation of the core-link format and /.well_known/core interface and perform HTTP POST operations to relay CoAP GET responses to a push-capable resource like a Cosm feed.

Sensor nets also may have unique discovery and location services that need to be mapped to special resources on the HTTP side or special data types.

Gateway Internal resources

The Gateway contains internal resources used for
  • Resource mapping
  • Protocol mapping
  • Resource caching
  • Security
  • Semantic discovery and linkage

Here is an example of HTTP-to-CoAP gateway resources using the SmartObject API

The SmartObject API provides resources useful for constructing gateways using any proxy pattern, with or without caching, while adding security and semantic web linkage. For a more thorough description of the SmartObject API resources, see the earlier posts in this blog.

Sensor Net Gateway

The SmartObject API may be implemented in a sensor net gateway that has multiple hardware interfaces, e.g. WiFi and some profile of 802.15.4 talking to smart sensors running CoAP.

Super Net Gateways

Some sensor nets provide integral gateways, e.g. "tag manager" with standard network interfaces but proprietary protocols. A SmartObject API gateway can be added to these networks for overall management and federation, and as a gateway to cloud services, e.g. Cosm, ThingWorx... In this case, only the LAN interface is needed.

Another use case for the super-gateway is CoAP, which can be routed on LANs. The CoAP network may be connected by a simple CoAP router or bridge via CoAP-over-IP to the internet gateway.

Application agents and software can run in the IoT gateway, which can also host local smartphone and browser-based apps and controls. This allows the system to operate locally without an internet connection if needed.

Gateway-as-a-Service (GaaS)

Finally, the gateway can be used as a per-user resource provided by a cloud service to interact with gateways, other cloud services, and user devices. SmartObject gateway instances can be created by a cloud service to encapsulate per-user resources that can be load balanced, replicated, backed up, etc. Since SmartObject API objects include software agents, the gateway pattern can be used as a front end M2M interface for IoT cloud services. Note that CoAP can be routed over the Internet and is an alternate protocol for M2M interaction.


The Internet of Things consists of a number of heterogenous sensor networks and disparate data sources. A gateway is used to connect these disparate protocols to a common RESTful M2M internet protocol.

The Smart Object API contains resources and methods to provide a framework for implementing IoT gateways. The reference implementation of the Smart Object API will provide a gateway with HTTP and CoAP interfaces, with caching proxy and core-link-format to Semantic Web link bridging.

The reference implementation of a Smart Object service consists of a set of Smart Object gateway instances, each having a separate virtual environment and user/owner encapsulation.

Other IoT gateway frameworks

The OSGi framework is of general interest as a standard way to plug in sensor net protocols, data models, software agents, and other modular software components. Many of the frameworks use OSGi.

There are a few IoT gateway framework projects underway in the EU IoT community.

ETSI - EU Telecommunication M2M Specification
FI-WARE - Future Internet Core Platform ( )
Eclipse M2M gateway (Eurotech)

Ericsson appear to be working on a gateway incorporating some or all of these specifications.

There seems to be a broad framework but no specific data models beyond the use of RESTful interfaces. The SmartObject API and gateway functions described above could be integrated into these frameworks and provide service level interoperability including semantic data discovery and linkage.

Further reading on CoRE and CoAP

Constrained Application Protocol

CoAp Feature Analysis

Observing Resources in CoAP

Best Practices for HTTP-CoAP Mapping Implementation

Sunday, September 9, 2012

Data Models for the Internet of Things #5, The Smart Object API in the IoT Infrastructure

Data Models for the Internet of Things #5, The Smart Object API in the IoT Infrastructure

Michael J Koster

The goal is to create a web ecosystem of sensingreasoning, and action around the Internet of Things.

The Smart Object API is a Semantic Web application for the Internet of Things. A RESTful web object encapsulation of semantic elements and real-time data properties, the Smart Object API provides pluggable live linked-data interaction between application software agents and IoT endpoints, sensors and user devices.

In the previous posts, I introduced the concept of a common data model for the Internet of Things based on emerging WWW technology and design patterns. I propose a Smart Object API as the basis of one such data model.

In this post I want to discuss how the Smart Object API would fit into the larger IoT infrastructure. There are a number of perspectives to consider, among them supporting service infrastructure, deployment patterns, integration with other stacks and standards, and specific build-out of the Smart Object pattern itself. 

For a reference model of the Internet of Things, I refer to the EU IoT Architecture Project's (newly revised) Architecture Reference Model, which is a broadly inclusive description of IoT architectures.

High-Level Infrastructure

For a view of the overall IoT infrastructure, and how the Smart Object API fits in, a good summary is in the IoT-Architecture document, D1.3 Architecture Reference Model, Figure 15, Functional View:

The Smart Object API defines a Virtual Entity, Application Interface, and IoT Service interface. The Smart Object framework will additionally define some common application agent patterns and a general service framework. 

The IoT Information model, figure 13 of the same document, further defines the Virtual Entity and it's component relationships. The Smart Object API has analogous relationships corresponding to it's SmartObject, Description, ObservableProperty, and the PropertyOfInterest classes.

The SmartObject class defines a VirtualEntity, ObservableProperty is an Attribute, PropertyOfInterest is a ValueContainer, Value is Instance of a Type, Descriptions are Metadata making up Service, Resource, and Device Descriptions.

 Deployment Patterns

The current deployment pattern for the IoT can be approximately characterized by the channel model described in figure 23 of the IoT ARM document:

The Smart Object API supports the concept of a Smart Gateway, which acts as a Smart Object proxy for devices on the Constrained Network, adding semantic descriptors and providing a service interface for the device representation on the Web.

Other Smart Object IoT endpoints include smart sensors, which have enough resources to interact using the service interface, and user client devices such as smartphones, which also have sufficient resources to interact using http and REST interfaces.  Smartphones can also act as ad-hoc gateways to connect to sensors over Bluetooth, USB, etc. in order to update Smart Object representations on the web. 

The CoAP protocol is a good example of a constrained protocol that is easy to integrate using the Smart Object API. CoAP defines a RESTful interface and structured resources using URIs. A Smart Object instance can be created on the gateway to act as a proxy for the CoAP resources, and a simple CoAP agent can interact with sensors on the constrained network to update the Smart Object properties and it's web app subscriptions.

There are some fixed-function constrained network gateways that support simple web interfaces. These can be easily extended using a service proxy running on a small computer like a $60 Android stick or $35 Raspberry Pi. The service proxy can interact with the gateway's fixed web interface and provide Smart Object wrappers for the attached sensors. It can provide a local service environment for application agents and direct user device interaction.

Finally, there are smart sensors with e.g. Wi-Fi interfaces that can interact directly on the Internet. These might also be though of as sensors with integrated Smart Gateways. An Arduino based sensor with a serial Wi-Fi module has enough resources to interact using a proper subset of the Smart Object API, enough to interact directly with a Smart Object proxy on a web service or another gateway. 

Integration with other Stacks and Standards

The Smart Object API is semantically identical both internally (program API) and externally (web API). This allows the API to be used as a library for web access to Smart Object instances, or by creating a local Smart Object instance that the application agent can synchronize sensor data to using subscriptions and data push.

The Agent can be used as a set of methods and handlers in a monolithic application program, or be deployed inside a service as an autonomous software entity that evaluates rules or algorithms in a data flow graph, synchronized by data updates from sensors, etc. 

Standard web formats will be used for the web interface documents. HTML, XML, JSON, and RDF are the underlying types with PropertyOfInterest data type being negotiated semantically.

The data model should be interoperable with the Semantic Web linked data formats. GET on Smart Object level URIs should by default point to the RDF describing the Smart Object to enable further interaction. 

The data model is fundamentally interoperable with CoAP, and allows simple transparent caching proxy implementation between CoAP and Smart Object API, using a CoAP Agent to manage the caching, subscription updates, etc. An instance of a Smart Object API can be created on a CoAP node if resources permit, by routing resources using URI-Path as per the CoAP Feature Analysis document.

Smart Object Service Framework

Some additional functions are needed to create a build-out infrastructure for a prototype Smart Object Service. 

The Smart Object itself can have some common patterns built in:

Methods for self-assembling Smart Objects from Descriptions. Structural RDF elements describe ObservableProperty instances and Agent Instances. These instances should be created from the RDF itself, allowing Smart Objects to create and replicate themselves from sensor and composite object descriptions.

A semantic discovery and linkage engine is needed to allow application agents to connect to ObservableProperty instances of other Smart Objects. Facilitates examining RDF descriptions and constructing URIs to interact with and create subscriptions to properties of interest.

Reference agents for common patterns, e.g. timer, PID controller. Usable as templates for more sophisticated agents. Pluggable agents.

Services need a few additional features for managing collections of Smart Objects:

RDF crawler, database, indexing and discovery engine with SPARQL interface. A service can crawl the RDF and build search indices for first order discovery. Services can be built for large-scale discovery and linkage by crawling other services and aggregating results using MapReduce.

API capability-by-method key management framework. Key generator API using user secret. Key lifetime management. 

URI router for path based routing to Smart Objects on service. Threaded web server interface. Facilitates scale-up of service to web volumes.


The Smart Object API is a Semantic Web Linked Data application for the Internet of Things (IoT). It consists of a URI-Object encapsulation of semantic and real-time data properties associated with features of interest. 

The Smart Object architecture roughly conforms to the Virtual Entity, the Information Model, and the Channel Model set out in the IoT-A Architecture Reference Model (IoT-A ARM).

Supports direct interaction between smart sensors, smart gateways, cloud/internet services, and user devices. Interaction uses standard web protocols and formats and is semantically a superset of the CoAP protocol.

Service framework is to include object creation from semantic metadata, semantic database, discovery, and linkage, API capability keys, and threaded server.


SPITFIRE: Towards a Semantic Web of Things

Pfisterer, Roemer, et. al. 

A Resource Oriented Architecture for the Web of Things
Guinard, Trifa, Wilde

Building Blocks of the Internet of Things: State of the Art and Beyond
Serbanati, Medaglia, Ceipidor

Linked Data - Design Issues
Tim Berners-Lee

The Scale-free nature of the Web
Tim Berners-Lee

CoAP Feature Analysis
Shelby, et. al. 

Constrained Application Protocol (CoAP) 
Shelby, et. al.

Architectural Reference Model for the IoT (updated)
Updated reference model for IoT v1.5, Internet-of-Things Architecture EC Project
Bassi, Giacomin reviewers
D1 3_Architectural_Reference_Model_updated.pdf