Data Flow Component

A Data Flow Component (DFC) is integral to data movement within the United Manufacturing Hub (UMH). DFCs can be: Protocol Converters, Data Bridges, Stream Processors or Custom Components.

Each DFC is based on benthos-umh, a fork of Benthos (now known as Redpanda Connect)

Types of DFCs

Protocol Converter

A Protocol Converter acts as a bridge between external components—such as PLCs, HTTP endpoints, or MES systems—and the Unified Namespace. It converts various protocols into the event-driven architecture of the Unified Namespace.

Data Bridge

A Data Bridge connects UNS Core Modules with each other. Examples include:

Bridging between MQTT and Kafka (and vice versa).
Connecting Kafka to PostgreSQL/TimescaleDB for our Historian feature.
Kafka-to-Kafka bridges between UMH instances.

Stream Processor

A Stream Processor consumes and produces data within the UNS. It can be used for:

Simple tag renaming or modeling.
Data enrichment.
Implementing triggers or alerts.

Custom Data Flow Component

A Custom Data Flow Component is a flexible Benthos container where you can freely specify:

Inputs
Processing logic
Outputs
Caches

Sonnländer increased asset productivity with OEE monitoring

Company Snapshot

Sonnländer, a subsidiary of EDEKA, is a leading producer of fruit juices and fruit-based products in Europe. With over 400 employees and 4 production facilities, it processes over 400,000 tons of fruit annually.

Challenges

Heterogeneous machine park
No live feedback from the shop floor
Internal resources need to focus on use cases, not infrastructure

Outcome

10-15%

Increased OEE

10-20%

Increased asset availability

>40%

Shorter root-cause problem solving

Austrian automotive supplier reduces costs with an open-source historian

Company Snapshot

A global automotive supplier specializing in advanced connectivity solutions for over 60 years. With over 6,000 employees, the company operates 12 highly automated plants worldwide.

Challenges

High costs of validating point-to-point integrations
Slow time to value
High license costs for MES & Historian

Outcome

>100k

Saved in license cost per site

97%

Reached in data compression rates

0%

Vendor lock-in in case of license changes

European energy producer increases implementation speed with Unified Namespace architecture

Company Snapshot

A major European energy company with a focus on renewable energy and innovative energy solutions. Employing over 75,000 people, it operates in multiple countries.

Challenges

Unstable internet connections
Limited throughput rate of OT systems
High reliability demands

Outcome

80%

Reduced integration time per use-case

60%

Reduced Mean Time To Detect (MTTD)

90%

Reduced integration maintenance

European energy producer increases implementation speed with Unified Namespace architecture

Company Snapshot

A major European energy company with a focus on renewable energy and innovative energy solutions. Employing over 75,000 people, it operates in multiple countries.

Challenges

Heterogeneous machine park
No live feedback from the shop floor
Internal resources need to focus on use cases, not infrastructure

Outcome

80%

Reduced integration time per use-case

60%

Reduced Mean Time To Detect (MTTD)

90%

Reduced integration maintenance

Features

Configuration Options

Inputs and Outputs
A comprehensive list is available on the Protocol Converter Page.
‍
Processing

Code-Based Processing: Use Bloblang or JavaScript for complex transformations.
Simple Operations: Implement functions like dedupe for message deduplication or "report-by-exception"
Advanced Enrichment: Utilize processors like http or sql for data enrichment.

Resources
‍Create and connect resources such as caches or buffers to assist in processing.

Lightweight and Fast

Minimal Footprint
Each DFC is a lightweight Docker container fully configured via a single YAML file. No states, large frameworks, or databases—just start it up, and it moves your data.

Flexible Deployment
Deploy a DFC/Benthos container standalone as a single binary on Windows, macOS, or Linux, or as a Docker container. Check out the releases on GitHub for more information.

Reliability

At-Least-Once Processing
By default, a DFC ensures at-least-once data processing. It does not acknowledge messages from its input until the result is successfully written to the output.

Quick Recovery
DFCs start up quickly and, when used with Kubernetes, can be rapidly rescheduled on another node if the current node fails.

Maintainability

Monitoring
If a container isn't starting or shows warnings/errors, the Management Console remotely monitors and alerts you.

Logging and Metrics
Access each DFC's logs and metrics directly from the Management Console.

More Resources

Management Console

Connect. Integrate. Analyze. Our platform links operations with a Unified Namespace message broker, while tracking historical data and delivering real-time shop floor KPIs and analytics.

Accessible

Unite all data in a single point with our real-time and event driven architecture: the Unified Namespace

Manageable

Gain full control and transparency over your IT and OT systems

Resilient

Built with scaleable & proven cloud native technologies on top of established automation standards

Learn more

Extend the Automation Pyramid with Unified Namespace today

Get started Book a call

Data Flow Component

Types of DFCs

Protocol Converter

Data Bridge

Stream Processor

Custom Data Flow Component

Sonnländer increased asset productivity with OEE monitoring

Company Snapshot

Challenges

Outcome

10-15%

10-20%

>40%

Austrian automotive supplier reduces costs with an open-source historian

Company Snapshot

Challenges

Outcome

>100k

97%

0%

European energy producer increases implementation speed with Unified Namespace architecture

Company Snapshot

Challenges

Outcome

80%

60%

90%

European energy producer increases implementation speed with Unified Namespace architecture

Company Snapshot

Challenges

Outcome

80%

60%

90%

Features

Configuration Options

Lightweight and Fast

Reliability

Maintainability

More Resources

Filtering and Sampling

Official Benthos Enrichment

Node-RED in Industrial IoT

Benthos-UMH Documentation

Benthos vs. Node-RED

Kubernetes Rolling Updates

Getting Started Guide & Requirements

Tutorial: Compression and Retention in UMH

Why we chose TimescaleDB over InfluxDB

Historians vs Open-Source databases - which is better?

The High Compression Deception: A Reality Check on Historians vs Open-Source Databases

Tools & Techniques for scalable data processing in Industrial IoT

Comparing MQTT Brokers for the Industrial IoT

Why designing your own IT / OT infrastructure is harder than you might think

Integrating UNS into Your Enterprise Architecture: An Architect's Guide

UNS: Strongest Architectural Proposal for Industry 4.0

Filtering and Sampling

Official Benthos Enrichment

Mastering Data Transformation with Benthos and Bloblang in 19 Examples

Node-RED in Industrial IoT

Our Take on OPC UA

Benthos-UMH Documentation

Benthos vs. Node-RED

Why we Developed Our Own Protocol Converter

Management Console

Accessible

Manageable

Resilient

Extend the Automation Pyramid with Unified Namespace today