Dr. Paddy Byers, Ably Realtime’s CTO, led a team that spent 50,000+ hours developing Ably’s data stream infrastructure supporting realtime data communication for the transportation industry.


Where do you see the greatest need for tech innovation in the transportation industries?

Much is written about ‘megatrends’ and how they affect the transport industry. What people tend to overlook, however, is the infrastructure requirements that will enable these trends. Autonomous vehicles and Mobility as a service (MaaS) transport solutions rely on huge amounts of data, reliably transferred, processed, and delivered in realtime. While 5G networks enable connectivity, they fail to build and maintain sub 100ms connections.

For the innovation curve to keep climbing, infrastructure is needed to allow for easy communication of live updates on transport systems for its end-users, and for developers to easily share and build on realtime APIs.


A lot has been happening in terms of autonomous vehicles, IoT enabled components, continuous connectivity. Where do you see changes happening most rapidly?

Just as the ‘tech industry’ ceased to be a legitimate concept when everything became tech, the proliferation of connected, data-emitting devices mean Internet of Things is no longer a ‘thing.’ In transport, data-emitting devices are the technology catalyzing transformative innovations all across the sector. Autonomous vehicles, P2P transport solutions, even Uber—all rely on IoT. Without connectivity though, IoT devices are useless.

Better connectivity—of which realtime infrastructure is a vital part—will supercharge the transformative potential of data-emitting devices in the mobility sector, enabling connections between modes of transport, transport providers, users of transport, controlled through a central transport grid.

An example of what can be done here is MaaS provider Moovit’s collaboration between TomTom and Microsoft. They use real-time data transfer to connect cars to the wider grid of public transport. About overcoming the lack of realtime data structure to connect transport modes, Moovit spokesperson commented, “If you can combine real-time and traffic information with the driving part of a journey, and then connect that directly to the real-time schedule of the transit system, then we can create a much more streamlined trip.” People are sensitive to disruptions in their journeys; what prevents people from taking an Uber or Lyft as the first part of their journey only, before using public transit, is that it can be very unpredictable. We are therefore trying to reduce the number of surprises; if we can connect it correctly and provide people with a reliable plan.

Uber’s adding information about other modes of transport to its journey-finding app—thanks to data shared from TfL—is part of the same phenomenon.

Realtime connectivity between transport modes is on the up, and as a result, we are seeing a more holistic approach to tackling issues facing public transport.


In what areas do you see resistance and slower adoption of tech in transport?

Transferring real-time data at scale comes with a host of technical complexity. Reliable data stream networks need to be continuously maintained, they are challenging to scale, and they need built-in mechanisms to deal with bursts in usage and in-built message ordering. Problematically, realtime data transfer relies on a highly fragmented ecosystem of protocols—to build multiple connections, developers need to develop many translation layers. REST APIs come with terms of use that state developers must ingest data, process it, and then republish it to integrate into apps and services. All this takes up valuable engineering time.

The points above mean that, despite huge developer demand, realtime APIs are still hard to handle.

TfL’s open data exemplifies the value of open data streams, and yet other cities have been slow to follow this example. By making realtime APIs easy to deploy, integrate, and distribute, the Ably Hub and Streamer have the potential to kick-start important innovations around publically available realtime data. Matthew O’Riordan Founder of Ably Realtime

As well as the financial barriers to entering the realtime data economy, obstacles to opening up data streams tend to be cultural. People and businesses are choosing to keep to technologies that lock-in existing commercial agreements around data, rather than adopting new technologies that enable a more outside-in approach.


Realtime data transfer is undoubtedly vital in autonomous driving, but why is it essential in other areas?

Transport for London’s decision to share live updates gave rise to hundreds of apps based on this data, including household names such as CityMapper, and a community of 13,000 developers building innovations based on real-time transport updates. A Deloitte study estimated revenue generated from Transport for London’s (TfL) decision to open up its data was worth £130m.

In Europe more widely, it’s been estimated that by using open data effectively, 629 million hours of unnecessary waiting time could be saved on the EU's roads and energy consumption cut by 16 percent.

As mentioned above, connected systems enable better, more holistic approaches to keeping a city moving, with positive effects on air quality, congestion, and other aspects of urban life.


What roadblocks do you see in implementing realtime data transfer in this industry?

Roadblocks tend to be cultural and technical. One thing is to reliably stream data at scale, bearing in mind the difficulties associated with message ordering, delivery guarantees, scalability, and so on. Another significant technical roadblock is the lack of unified streaming protocols. For example, with many IoT devices relying on the MQTT protocol, good translation layers are needed if this data is to be transported, processed, and leveraged. For developers, this represents an obstacle in the innovation process. It would be much easier if data streams could be accessed, shared, and built upon in a protocol-agnostic environment.

Just as API management tools made it easy for companies to exchange and share data outside their organizations, realtime API Management tools make multiple streaming connections much easier to manage and aggregate.

Realtime transport data can be usefully employed to create a holistic city-wide and multi-modal transport system. Data transfer systems must avoid intermediaries, data congestion, and unnecessary complexity, and emulate models that allow data to be sent directly from the source to the end-user. Just as with real transport systems, effective realtime data transport relies on closed systems guaranteeing capacity and availability while providing the shortest possible data transfer route.


How would real-time data transfer be better than current electronic data exchanges like EDI?

EDI and realtime data transfer are essentially two different things. When it comes to mobility, realtime data transfer is what works best. While EDI is meant for the exchange of business documents, in realtime data transfer, the data is just a message, typically 16 - 256 kb. EDI documents are typically user-generated while realtime messages emerge from a software application or sensor data.


What benefits do you expect to see five years from now in terms of tech innovation and implementations?

Consumer-end, as realtime transport data becomes more available, we can expect to see a proliferation of time-saving apps and services. For example, in New York, information is now available on airport websites and terminal monitors for passengers traveling to JFK, LaGuardia, Newark Liberty, New York Steward. In New York City passengers can get detailed live information not just on transport time, but also on seat availability. As the Uber example shows, better—more centralized—ways of exchanging real-time data streams will make integrated journeys more feasible.

AVs, expected to replace the world’s 1.4 billion cars within ten years, according to some sources, exemplify the disconnect caused by infrastructure failing to keep up with technology. The perfect storm of the maturation of IoT, AI, and ML systems, AVs rely heavily on real-time data transfer, with each car transferring 4,000 GB of data each day.

In a city like London, if its 2.6m vehicles become driverless, realtime infrastructure requirements (just for running cars) at 10,400,000,000 GB data per day and 120,370 GB per second, as a minimum, a mission-critical requirement for keeping traffic moving.

We already see significant investment in connectivity infrastructure (notably 5G) in major cities. As more investment in data transfer infrastructure is necessitated by other tech developments, we will see multi-modal transport systems and innovation from real-time APIs.


What recommendations do you make to IT executives in the transport industry as they look to gain advantages over their competition?

Realtime features and functionality are rapidly becoming basic requirements for market participation. Platforms and tools are only just emerging to ease the transition from static data exchange to realtime data exchange. As we move forward and investment requirements fall, more transport companies will provide realtime. The time to act is now for those IT leaders who want to carve out a defensible position in the future of the transport industry.

As in other industries, the way to ‘defend’ is actually by opening up and letting innovation in.


»Read next, on Emerge: