How Data Gets Into the PI System: Interfaces, Adapters and MQTT

Meta description: Practical guide to PI data ingestion: compare Interfaces vs Adapters, MQTT/UNS patterns, OPC UA, OMF modelling, data quality, and migration.

Why ingestion is more than “getting tags in”

PI ingestion is where OT realities meet your PI architecture: noisy signals, legacy protocols, security zones, buffering, naming standards and the inevitable “it worked in the lab” surprises.

Well‑designed ingestion produces trustworthy, scalable time-series data with clear ownership and predictable operations. Poor ingestion yields brittle edge estates, inconsistent tag design and late-emerging data quality issues that break reports and analytics.

This guide explains the main approaches—PI Interfaces, PI Adapters and MQTT-based patterns—and how they relate to OPC UA, OMF and modern modelling.

If you want the wider context first, see the core PI System architecture overview: Designing a Scalable and Resilient PI System Architecture

---

High-level components of ingestion

Essential moving parts in any PI ingestion pipeline:

• Data source: PLC/DCS/SCADA, historian, analyser, application, MQTT broker.
• Edge connector: PI Interface, PI Adapter, custom app using PI Web API/AF SDK, etc.
• Transport and buffering: network path, store‑and‑forward, retry behaviour.
• PI endpoint: typically the PI Data Archive, often with AF for context.
• Security & identity: service accounts, certificates, firewall rules, trust lists.
• Operational tooling: monitoring, patching, upgrades, troubleshooting.

Classic deployments used Interfaces on a Windows interface node. Modern patterns use Adapters on edge hosts (often container-friendly) and sometimes MQTT upstream. For day‑to‑day ownership and support boundaries, see Running the PI System Day-to-Day: A PI Admin’s Playbook

---

Interfaces vs Adapters: practical differences

Both bring data into PI. The distinction is about runtime model, protocol focus and operational fit.

PI Interfaces (classic)

• Windows services that connect via a source-specific driver or API and write directly to the PI Data Archive.
• Usually installed on a dedicated interface node.
• Mature operational practices; buffering commonly via PI Buffer Subsystem (PIBufSS).
• Best when you have stable legacy systems (e.g. OPC DA) and established interface-node procedures.
• Trade-offs: Windows footprint, patch cadence, and complexity with legacy COM dependencies.

PI Adapters (modern)

• Designed for modern OT/IT patterns; often use OMF for ingestion and support flexible deployments (Windows or Linux).
• Better support for OPC UA, MQTT and structured metadata.
• Fit centralised management and repeatable rollouts; suit containerised edge hosts.
• Trade-offs: require upfront naming, modelling and topic conventions; skills shift to data modelling and edge operations.
• Some legacy endpoints still need Interfaces or gateways.

Decision lens:

• OPC DA + Windows-centric → Interface often pragmatic.
• Strategic OPC UA or MQTT + repeatable edge pattern → Adapter usually better.
• Migrations commonly run both for a while—plan duplicate suppression and cutover controls.

For sizing and tuning, bookmark: Keeping PI Fast, Stable, and Predictable at Scale

---

MQTT and the Unified Namespace (UNS)

What MQTT is

• Lightweight publish/subscribe protocol: devices publish telemetry to a broker; consumers subscribe.
• Decouples producers and consumers and avoids point‑to‑point polling.

What UNS means

• An architectural pattern: a shared topic hierarchy reflecting enterprise/site/area/asset structure.
• Emphasises consistent naming and message structures so multiple consumers can reuse streams.
• Supports event‑driven data and explicit context.

UNS is powerful but requires governance:

• Topic standards, versioning, ownership and lifecycle management.
• Decisions on what lives in MQTT versus what is modelled in PI AF.
• Handling of state, quality and backfill behaviour.

Practical rule: MQTT moves data; PI (especially AF) organises and enables use. Don’t try to make one replace the other without a clear rationale.

---

PI Adapter for MQTT: overview and cautions

The PI Adapter for MQTT subscribes to broker topics, interprets payloads and forwards data to PI using an OMF-based path. Common architectures:

• Edge → broker → PI
• Broker in a DMZ to mediate cross‑zone access
• Local broker per site with central aggregation for resilience

Design choices:

• Topic hierarchy and naming stability.
• Payload format: flat vs structured, timestamp inclusion, quality fields.
• Security: broker authentication, TLS, certificates, client IDs.

Operational gotchas:

• Silent topic changes break subscriptions.
• Retained messages can replay unexpectedly.
• Publisher clock drift causes out‑of‑order timestamps.
• Broker persistence and retention settings determine realistic backfill.

Treat MQTT ingestion as a product: document, monitor and test failure scenarios (WAN loss, broker restart, credential expiry).

---

OPC UA ingestion patterns

OPC UA is platform-neutral, secure and more expressive than COM-based OPC. Typical patterns:

Pattern 1 — Direct OPC UA to PI (Adapter)

• PI connector acts as an OPC UA client, browses/selects nodes and streams data with buffering and reconnection logic.
• Pay attention to subscription modes (sampling vs publishing), stable identifiers and namespace changes.

Pattern 2 — OPC UA gateway

• Terminates UA in a boundary zone and provides a stable client endpoint for PI.
• Simplifies certificate trust but adds an extra component to monitor (and a potential single point of failure).

Pattern 3 — OPC DA with UA wrappers

• Useful as a migration bridge but expect DA-era timestamp/quality quirks and multi-layer troubleshooting.

---

OMF and modelling implications

OMF (OSIsoft Message Format) structures time-series data and metadata. Beyond payload schema, it drives modelling decisions:

• Types: what a measurement represents.
• Containers: logical groupings/streams.
• Metadata: units, descriptions, contextual fields.

Why it matters:

• OMF surfaces decisions about units, ranges and consistent concepts across assets.
• These choices affect AF template strategy, searchability and migration complexity.
• If you are light on AF today, ingestion projects are the right time to adopt templates—retrofitting context later is costly.

---

Data quality: duplicates, late data and out‑of‑order events

Most ingestion issues stem from upstream behaviour and connector handling.

Duplicates

• Causes: retries after outages, buffered resends, MQTT retained messages, overlapping point sources.
• Mitigations: clear stream identity (point source/topic mapping), avoid parallel writes during migration, understand connector resend windows.

Late and out‑of‑order data

• Causes: edge buffering during outages, clock drift, burst releases on reconnection, multiple publishers.
• Mitigations: decide where timestamps originate, enforce time synchronisation (NTP/PTP), validate ingestion handling of out‑of‑order events.

Quality flags and semantics

• Protocol conversion can flatten quality codes. Decide whether to preserve, substitute or suppress bad quality.
• Watch for semantic quality issues (wrong scaling or units) — these are modelling/commissioning problems.

For throughput and backfill performance, see: Keeping PI Fast, Stable, and Predictable at Scale

---

Security and zoning

Ingestion is a security boundary. Key practices:

• Minimise inbound firewall rules; prefer defined, monitored paths.
• Treat edge nodes as high risk: least privilege, patching, service isolation.
• Use certificate-based trust where suitable (OPC UA, TLS-enabled MQTT).
• Document identities: accounts, credential locations and rotation processes.

Security is not “set and forget”: certificate expiry and credential rotation cause common outages.

See Securing the AVEVA PI System in Modern Enterprise Environments

---

Migrating legacy ingestion

Most sites must modernise incrementally. Recommended steps:

1. Inventory

• Record interfaces, versions, nodes, point sources, scan classes, buffer config, tag counts and known issues.

1. Define modernisation targets

• Examples: OPC DA → OPC UA subscriptions, MQTT + broker, AF-template modelling, reduced Windows footprint.
• Be explicit which outcomes matter: lower ops overhead, better security, improved modelling or resilience.

1. Run parallel briefly and safely

• Use a shadow namespace for the new path.
• Validate data equivalence with defined tolerances (timestamps included).
• Plan cutover windows and rollback steps.

1. Treat naming and modelling as a workstream

• Map old tags to new attributes, set naming standards, govern changes and communicate impacts.

1. Decommission with evidence

• Confirm buffers are empty, no pending backfill, and no consumers rely on the old path. Update documentation and monitoring.

---

Other integration patterns

Additional methods that appear in the field:

• Custom apps writing via PI Web API or AF SDK for derived events or business data.
• Historian-to-historian links for consolidation.
• File-based batch ingestion (lab data, exports) — requires careful timestamp and quality handling.

These are valid trade-offs; avoid unowned integration paths that become mission critical.

---

Common failure modes and quick cues

Symptoms and likely causes:

• Flat-lined tags across many points: connector stopped, buffering not flushing, network or credential/certificate failure.
• Intermittent gaps: scan-class overload, CPU contention on edge node, broker disconnects, OPC UA session drops.
• Bursty backfill: expected store‑and‑forward behaviour but inadequate downstream sizing.
• Duplicate-looking spikes: reconnect resend combined with compression and chattering signals.
• Wrong units/scaling: mapping/configuration error.

For operational context, revisit: PI Administrator runbook: daily, weekly, and monthly operational checks

---

When to bring in specialist help

Engage specialists for multi‑site standards (OPC UA/MQTT), large legacy migrations, or setting up a UNS and governance. Short, focused engagements typically deliver the fastest wins in architecture, naming/modelling and cutover planning.

Browse integrators and consultants: https://piadmin.com/directory