The Logic
Translating video into process metrics. The surface of a flotation bank provides key indicators of air rate, reagent regime, and hydrodynamics. Computer vision transforms these pixel-based cues into actionable numbers, ensuring continuous and deterministic monitoring without the need for intrusive hardware.
A Soft Sensor is a virtual instrument: it estimates hard-to-measure variables from other signals. Here, the “signal” is a video stream of flotation froth. The estimator is a vision pipeline that detects and segments individual bubbles in each frame, tracks them, and computes features over time. The output is a stream of real-time analytics, including Bubble Size Distribution (BSD), bubble velocity via Optical Flow, and Process Stability indicators (variance of BSD, surface renewal rates, and coalescence frequency).
- Input. RGB video from a ruggedized camera positioned to view the froth launder.
- Processing. Deep Neural Networks (DNN) for Instance Segmentation (per-bubble masks), temporal association, and physics-aware post-processing.
- Output. Metrics including BSD (e.g., D10/D50/D90), bubble speed, froth mobility, and stability proxies. Signals are timestamped, quality-scored, and available for plant historians and dashboards.
This is the foundational step toward Automation and control: once bubbles become time-aligned, machine-readable signals, you can correlate them with cell setpoints and downstream metallurgy, fit dynamic models, and ultimately actuate control (e.g., air rate or frother dosage) safely.
Technical Challenge
Why conventional monitoring is unstable. Traditional approaches rely on Classical Image Processing such as intensity Thresholding, edge finding, and Blob Analysis to infer bubbles from bright rings caused by specular reflections at the air–liquid interface. These heuristics break down in real plant conditions because reflection-based detection assumes a near-orthogonal top-down camera view that often does not hold in practice, while lighting changes from cloud cover, lamp aging, or slurry color shifts cause fixed thresholds to drift. The approach is also highly angle-dependent, as small camera tilts change Fresnel reflections and turn rings into arcs, making the same bubble appear different across frames. Coalescing froth further leads to object loss through merged highlights, while spray and splashes generate spurious blobs. The result is unstable counts, biased size estimates, and metrics that cannot be trusted for control.
Our Approach
From heuristics to learning. We replace reflection heuristics with Deep Learning detectors that delineate each bubble by its mask, not by brightness artifacts.
- Deep Neural Networks (DNN). Backbone networks learn froth morphology directly from data, not from hand-tuned thresholds. We use architectures such as Mask R-CNN and YOLOv8-seg to output pixel-accurate masks per bubble. Masks survive lighting and viewing-angle changes better than ring detectors because they leverage shape, context, and texture.
- Transfer Learning. We initialize from models pre-trained on large-scale segmentation corpora, then fine-tune on froth imagery to accelerate convergence with limited labeled data.
- Data Augmentation. We randomize illumination, contrast, scale, blur, and camera tilt; we inject synthetic glare and droplets to immunize the model against nuisance factors.
- Inference Optimization. We prune and quantize the models for edge targets (e.g., INT8 on an embedded GPU), batch strategically (micro-batches), and pin memory to sustain low-latency throughput.
Why per-bubble segmentation matters. Control logic needs correct geometry (area, perimeter, roundness) and topology (merge/split events). Reflection-only detectors miss dark bubbles, double-count specular rings, and cannot distinguish partial occlusions. Segmentation gives true object detection with object tracking, continuous, stable inputs for analytics.
Advanced Post-processing
Raw detections are not yet plant-ready, so we apply deterministic post-processing to stabilize signals. Custom Non-Maximum Suppression (NMS) uses mask-overlap IoU to handle tightly packed bubbles without over-suppression. Thresholding logic removes low-score masks and enforces size plausibility bounds from site commissioning. Specular Reflection handling down-weights glossy regions using a glare-map and merges highlight-induced duplicates. Feature Extraction computes per-bubble geometry and frame-level texture features to proxy froth stiffness, while object completeness filtering removes partial bubbles to avoid BSD bias.
Computer Vision for Industrial Analytics
Metric derivation. With clean per-bubble tracks:
- Bubble Size Distribution (BSD). We produce rolling histograms and percentiles (D10/D50/D90) per window. BSD is normalized to camera scale. In extended configurations, stereo baselines or single-view scale factors can be applied where available.
- Process Stability Indicators. Low-frequency variance of BSD, burstiness of merge events, and stationarity of rise speeds quantify the froth regime. A rising variance with a constant air rate flags a reagent or feed change.
- Optical Flow (for velocity estimation). Surface motion is estimated from temporal bubble displacement. In advanced configurations, surface velocity can be estimated using optical flow across successive frames to handle noise, partial occlusions, and lighting variation.
- Real-time Analytics. All metrics are emitted at 1–5 Hz with quality flags and are suitable for video analytics dashboards and historians.
Why physical sensors can’t do this, a conventional probe measures a point signal (pressure, conductivity). Froth metrics are inherently spatial and statistical: BSD, topology, and surface transport require seeing many bubbles across an area and over time. That constraint is precisely what vision and only vision satisfies.
Analytics for Process Control
Scalability to a closed loop builds on the demo, which provides validated sensing but not full control. The next step is to apply Model Predictive Control (MPC) to link setpoints with Bubble Size Distribution and stability targets, expose metrics via OPC UA to distributed control systems (DCS) and supervisory control and data acquisition systems (SCADA), and enforce interlocks so control actions run only within safe and confident bounds. This follows the standard soft sensor path of validating metrics, closing the loop gradually, and auditing results against metallurgical KPIs.
Deployment & Infrastructure Safety
Deployment and infrastructure safety are shaped by industrial constraints. Real-time inference runs on edge AI hardware close to the flotation cell to avoid WAN jitter and keep raw video inside the OT zone. The demo targets a dedicated compact edge AI node, such as CamBox→, which provides GPU acceleration while operating within strict thermal and power limits. The vision node is isolated from critical control networks through a DMZ. Only whitelisted OPC UA metrics are allowed to egress. There is no inbound control path to the camera. Firmware and computer vision camera drivers are pinned and signed. This keeps latency predictable and the control network uncompromised.
Conclusion
We have detailed a direct framework for converting video streams into machine-readable signals. By segmenting individual bubbles with Instance Segmentation (e.g., Mask R-CNN, YOLOv8-seg) and applying rigorous post-processing (mask-aware NMS, IoU, and precise Feature Extraction), the vision pipeline functions as a reliable Soft Sensor. This transition allows plants to replace subjective operator judgment with objective metrics, supporting Real-time Analytics today and MPC implementation tomorrow, deployed securely on existing Edge Computing nodes.
It’s time to work smarter
Evaluating computer vision for froth flotation?
We can help outline constraints, trade-offs, and practical next steps.
