Multimodal sensing glossary: 40 terms engineers actually use

A multimodal sensing glossary is useful because the field sits between disciplines. A mechanical engineer, an electronics designer and a data scientist can sit in the same meeting, use the word "bandwidth" and mean three different things. The definitions below are the ones we use in projects and in our writing.

How to use this multimodal sensing glossary

The 40 terms are grouped by topic rather than alphabetically, because related terms are easier to understand together. Each definition is one to three sentences. Where a formal definition exists in a standard, we point to it; for condition monitoring, the formal reference is ISO 13372:2012, which defines terms used in condition monitoring and diagnostics of machines.

If you are new to the subject, start with our introduction to what multimodal sensing is, then come back here as a reference.

The groups follow the path of a measurement. A physical effect is picked up by a sensor of a particular modality, turned into numbers by an acquisition chain, given a time stamp, written into a recording and finally evaluated by a model. An error at any stage propagates downstream: a model cannot recover a frequency the anti-aliasing filter removed, and a label cannot be trusted if the time stamp it relies on has an unknown offset. Reading the glossary in order is therefore a quick way to check a system design for weak links.

We have kept the definitions practical. Where a term has a stricter meaning in a standard or a specification, use that meaning in contracts and test reports, and use these definitions to get everyone in the room talking about the same thing.

Sensing modalities and sensors

These terms describe what is being measured and with what. When a project team argues about which sensor to add, the disagreement is often really about which physical quantity carries the fault, so it helps to name the modality first and the product second.

Multimodal sensing

Multimodal sensing is the acquisition and joint evaluation of signals from sensors that measure physically different quantities on the same object, such as light, infrared radiation, distance, sound and vibration. Several sensors of the same type form a multi-sensor system, not a multimodal one.

Modality

A modality is a type of signal defined by the physics behind it, for example reflected light, emitted infrared, time of flight, airborne pressure waves or surface acceleration. Two modalities are useful together when they fail in different ways.

Global shutter

A global shutter exposes all pixels of an image sensor at the same moment and then reads them out. It avoids the skewed edges that rolling-shutter sensors produce on fast-moving objects, which matters on conveyors, rotating parts and process monitoring.

Radiometric thermal imaging

Radiometric thermal imaging measures emitted long-wave infrared radiation and converts it into a temperature value for every pixel. Non-radiometric thermal cameras show relative heat patterns only, without calibrated temperature values.

Emissivity

Emissivity is the ratio of the infrared radiation a surface emits to that of an ideal black body at the same temperature. Low-emissivity surfaces such as bare, polished metal emit little and reflect a lot, so a thermal camera can read them wrongly unless emissivity and reflections are compensated.

Direct time-of-flight (dToF)

Direct time-of-flight is a depth-sensing method that emits short light pulses and measures the time until the reflection returns, using fast detectors such as single-photon avalanche diodes (SPADs). It gives a coarse but robust distance map that works on surfaces without texture and in the dark.

Event camera

An event camera is an image sensor whose pixels independently report changes in brightness as they happen, instead of capturing full frames at a fixed rate. The survey by Gallego and colleagues describes temporal resolution on the order of microseconds and a dynamic range of 140 dB, compared with about 60 dB for conventional cameras.

Measurement microphone

A measurement microphone is a microphone with a specified, flat frequency response and known sensitivity, built for quantitative acoustics rather than speech. Measurement capsules can reach well into the ultrasonic range and are often specified for a particular sound field, such as free field.

Airborne ultrasound

Airborne ultrasound is sound in air above the human hearing range of about 20 kHz. It carries friction, compressed-air leaks and electrical discharge, and it is less affected by the low-frequency background noise of a factory hall.

MEMS accelerometer

A MEMS accelerometer is an acceleration sensor made with micro-electromechanical structures on a silicon chip. MEMS parts are compact and cost-effective for condition monitoring. Depending on the part, their 3 dB bandwidth ranges from a few kilohertz to about 10 kHz and beyond (the ADXL1002 specifies 11 kHz), often with a higher noise floor than piezoelectric (IEPE) accelerometers.

Signal acquisition terms

These terms describe how a physical signal becomes numbers. They decide what a channel can see before any analysis begins.

Sample rate

The sample rate is the number of samples a converter takes per second, given in samples per second (SPS) or hertz. It limits the highest frequency that can be represented, but it says nothing about whether the sensor itself responds to that frequency.

Nyquist criterion

The Nyquist criterion states that the sampling frequency must be at least twice the highest frequency contained in the signal. In practice, the ADI tutorial MT-002 suggests starting with 2.5 to 4 times the highest frequency of interest to leave room for the anti-aliasing filter.

Aliasing

Aliasing is the folding of signal components above half the sample rate into lower frequencies, where they become indistinguishable from real signals. Once a signal is aliased, no later processing can undo it.

Anti-aliasing filter

An anti-aliasing filter is a low-pass filter placed before the converter to attenuate components above the band of interest. Its steepness and the sample rate are designed together.

Programmable-gain amplifier (PGA)

A programmable-gain amplifier is an amplifier whose gain can be switched in steps, so that small signals use the converter's full range and large signals do not clip. It is useful when signal amplitude varies widely between machines or operating conditions.

ADC resolution

ADC resolution is the number of bits an analogue-to-digital converter uses per sample, which sets the number of distinct output levels. More bits allow smaller signal changes to be resolved, provided noise in the rest of the chain is low enough.

3 dB bandwidth

The 3 dB bandwidth is the frequency range over which a sensor's or amplifier's output stays within 3 dB of its nominal sensitivity. Outside it the response deviates: amplifiers and filters roll off, while accelerometers usually rise towards their resonance. The ADXL1001/ADXL1002 data sheet is a typical case: its 11 kHz limit is set by an increase in response near the 21 kHz sensor resonance, so the 3 dB point is a gain in sensitivity, and the response stays within 5 % only up to 4.7 kHz.

Mounted resonance

Mounted resonance is the frequency at which a sensor, together with its mounting, resonates and amplifies the signal strongly. Measurements are normally restricted to a band well below it, and the mounting method can shift it considerably.

Time and alignment terms

Time terms explain why data from different sensors cannot simply be laid side by side. Our article on time-stamping multimodal sensor data covers them in practice.

Time stamp

A time stamp is a value attached to a sample, frame or event stating when it was captured according to a specific clock. A time stamp without a stated reference clock and uncertainty is of limited use for multimodal analysis.

Clock offset

Clock offset is the difference between a device's clock and a reference clock at a given moment. RFC 5905, the NTPv4 specification, estimates it from four time stamps exchanged between two machines.

Clock drift

Clock drift is the change of clock offset over time, caused by an oscillator running slightly fast or slow. The time error grows in proportion to the fractional frequency error and the elapsed time.

Jitter

Jitter is the short-term variation of time stamps or signal edges around their ideal positions. It limits how precisely individual events can be placed in time, even when average offset is well known.

Monotonic clock

A monotonic clock is a system clock that never jumps backwards or steps when the wall-clock time is corrected. Measurement time stamps are best taken on a monotonic clock, with its relationship to calendar time recorded separately.

Precision Time Protocol (PTP)

The Precision Time Protocol, defined in IEEE 1588, is a network protocol for aligning clocks in measurement and control systems. It achieves its precision largely by time-stamping messages in network hardware rather than in software.

Data and recording terms

These terms describe how recordings are organised so they can be trusted and reused. They matter as soon as data is used for model training or for quality records.

Manifest

A manifest is a file that describes a recording: which streams it contains, their formats, sample rates, sensor settings, software versions and file locations. A versioned manifest records how this description changes over time.

Segment hash (SHA-256)

A segment hash is a cryptographic checksum, such as SHA-256, computed over a fixed portion of a recording. If a single bit of the segment changes later, its hash no longer matches. Provided the reference hashes are stored separately or signed, and the manifest lists every segment, this shows whether data has been altered, truncated or removed.

Append-only journal

An append-only journal is a log to which entries can be added but never changed or removed. It provides an auditable history of what happened during a recording, including gaps, restarts and configuration changes.

NWB (Neurodata Without Borders)

NWB is a standardised, self-describing data format that packages time series data and metadata together according to a schema, as described in the NWB documentation. It originated in neurophysiology but suits other multichannel time-series data well.

JSON Lines (JSONL)

JSON Lines is a text format in which every line is one valid JSON value, encoded in UTF-8. It is well suited to event logs and label records, because files can be appended to and read line by line.

Tiered retention

Tiered retention is a storage policy that keeps different kinds of data for different periods, for example raw high-rate data for days, features and events for months and summary results indefinitely. It keeps storage bounded without losing the evidence that matters.

Model and evaluation terms

These terms describe how signals become decisions. The ladder from anomaly detection to severity grading is covered in more depth in our article on anomaly detection, classification and severity grading, and labelling for each level in labelling multimodal industrial datasets.

Sensor fusion

Sensor fusion is the combination of signals from several sensors into one estimate or decision. In multimodal systems, fusion is what turns independent physical observations into a single, better-supported conclusion.

Early and late fusion

Early fusion combines raw or lightly processed data from several modalities before modelling; late fusion runs separate models per modality and combines their outputs. Feature-level fusion, which combines extracted features, sits between the two and is the most common choice in industrial systems.

Feature extraction

Feature extraction is the computation of compact, informative values from raw signals, such as RMS, kurtosis, band energies, spectral peaks or temperature differences. Good features reduce data volume and make models easier to train and explain.

Anomaly detection

Anomaly detection identifies data that deviates from a learned model of normal behaviour, without needing examples of every fault. It is usually the first rung of the model ladder because normal data is plentiful and fault data is rare.

Severity grading

Severity grading assigns a detected fault to a graded level of seriousness, agreed with maintenance or quality teams. It answers "how bad is it?" and needs labelled examples at several severity levels.

ONNX

ONNX (Open Neural Network Exchange) is an open format for representing machine learning models with a common set of operators, according to onnx.ai. It allows a model trained in one framework to be deployed with a different runtime on edge hardware.

Condition monitoring terms

These final terms connect multimodal sensing to the established practice of machine condition monitoring.

Condition monitoring

Condition monitoring is the acquisition and processing of information that indicates the state of a machine over time. ISO 13372 provides the formal vocabulary, and ISO 17359 sets out general procedures for a monitoring programme.

P-F curve

The P-F curve plots a machine's condition over time from the point where a potential failure (P) becomes detectable to the point of functional failure (F). The NASA Reliability-Centered Maintenance Guide notes that the point of initial degradation and the point of initial detection rarely coincide, which is why earlier-detecting modalities are valuable.

Signal health

Signal health is an assessment of whether a sensor's data can be trusted, covering dropouts, saturation, implausible noise levels and loose mounting. A system that checks signal health can raise a sensor alert instead of a false machine alarm. In our cognitive sensors, signal health and dropouts are flagged at the edge.

Edge AI

Edge AI is the execution of machine learning models on hardware located at or near the machine, rather than in a data centre or cloud. It reduces data transfer and latency and keeps raw production data on site.

How NeuroControls uses these terms

We use this vocabulary across our documentation and projects. Our platform combines vision, thermal, depth, sound and vibration on a hub with dedicated nodes, time-stamps every stream on the hub and records each node's clock offset with its uncertainty. Recordings carry a versioned manifest, per-segment SHA-256 hashes and an append-only journal, with exports to NWB, MKV/HEVC, JSONL and CSV through our recording software.

Custom models trained by our engineers go from feature extraction and anomaly detection to fault classification and severity grading, delivered as ONNX. The multimodal sensing solution page shows how the pieces fit together.