Transient Detection (Spectral Flux)

Author

Affiliation

Cam

Anauseam

If an audio frame successfully passes the time-domain silence gate, the pipeline transitions into the frequency domain to evaluate the signal’s temporal stability. This phase constitutes States 1 and 2 of the Gatekeeper’s internal state machine: ATTACK and TRANSIENT.

Hammer Strikes

In acoustic pianos, the onset of a note is initiated by a felt hammer striking the strings. This percussive impact creates a violent, chaotic burst of broadband acoustic energy known as a transient.

During this brief attack phase, the physical string is still establishing its vibrational modes. As a result, the audio signal lacks a clear, stable fundamental frequency (f_0) and is heavily saturated with inharmonic noise. If downstream DSP engines attempt to extract pitch or measure inharmonicity during this window, the mathematical results will be highly erratic or completely invalid.

Normalized Half-Wave Rectified Spectral Flux (NHWRSF)

To isolate and flag these percussive strikes, the pipeline calculates the Normalized Half-Wave Rectified Spectral Flux (NHWRSF). Unlike simple time-domain amplitude tracking, NHWRSF measures the exact proportion of new acoustic energy introduced across a specific frequency band from one frame to the next.

This algorithm relies on three critical DSP mechanisms to guarantee accuracy on a piano signal:

• Phase-Blind Magnitude: It evaluates only the absolute magnitude of the frequency bins, ignoring phase. This prevents the natural phase-drifting of piano unisons from falsely registering as a new transient.
• Half-Wave Rectification: By mathematically discarding negative energy differences, the algorithm solely tracks increases in spectral energy. It triggers exclusively when a hammer strikes a string, completely ignoring the rapid drop in acoustic energy when a damper falls.
• Dynamic Normalization: The raw flux is divided by the total energy of the current frame. Because the output is a dimensionless ratio rather than an absolute amplitude measurement, a single threshold works flawlessly whether a key is struck pianissimo or fortissimo.

Given a frequency spectrum X with K frequency bins, the NHWRSF at the current frame \ell is defined as:

\text{NHWRSF}_\ell = \frac{\sum_{k=k_{\text{start}}}^{k_{\text{end}}} \max(0, |X_\ell[k]| - |X_{\ell-1}[k]|)}{\left( \sum_{k=k_{\text{start}}}^{k_{\text{end}}} |X_\ell[k]| \right) + \epsilon}

Where:

• k_{start} and k_{end} define the acoustic band of interest (e.g., isolating the core piano frequencies while bypassing sub-bass rumble and ultra-high frequency hiss).
• |X_\ell[k]| is the magnitude of the current frame’s spectrum at frequency bin k.
• |X_{\ell-1}[k]| is the magnitude of the previous frame’s spectrum at frequency bin k.
• The \max(0, ...) function performs the half-wave rectification.
• \epsilon is a mathematically negligible constant (such as 10^{-6}) added to prevent division by zero during absolute digital silence.

Below is a simulation of the NHWRSF calculation per every frame of an simulated piano note. One can clearly see the spectral flux confidently spike at the onset of the note, while remaining relatively low during the decay phase.

NoteSimulation Notes

1. Most programing languages start their indexing at 0, so the first frame is at index 0, the second frame is at index 1, etc. Thus, the slider starts at frame one to capture the initial difference.
2. In reality the frequency domain would show the peaks of the harmonics along with the fundamental. In this simulation, however, we only accounted for the fundamental frequency for simplicity.

TipFrequency Bins

For the uninitiated, the DFT/FFT¹ decomposes a time-domain signal into a series of discrete frequency bins. When you sample a continuous-time signal into a discrete-time signal x[n] of length N, the DFT maps those N time-domain samples into N discrete complex frequency-domain samples, X[k].

The DFT is defined formally as:

X[k] = \sum_{n=0}^{N-1} x[n] e^{-j 2\pi k n / N}

Here, k is an integer ranging from 0 to N-1. This integer k is the bin index. It does not represent a frequency in Hertz intrinsically; rather, it is a mathematical index representing an integer number of cycles that fit perfectly within the analysis window of N samples.

Frequency Resolution (\Delta f)

To map the bin index k back to a real-world physical frequency (f_k) in Hertz, we must introduce the system’s sample rate (f_s). The relationship is defined by the bin width, or better known as the frequency resolution (\Delta f):

\Delta f = \frac{f_s}{N}

Each bin represents a frequency band of width \Delta f, and the center frequency of bin k is evaluated as:

f_k = k \cdot \Delta f = k \cdot \frac{f_s}{N}

For example, if your pipeline processes audio with a sample rate of 44,100 Hz and a frame (FFT) size of N = 2048, your frequency resolution \Delta f is exactly 21.53 Hz. Therefore, bin k=1 represents the energy centered at 21.53 Hz, bin k=2 represents 43.06 Hz, bin k=3 represents 64.59 Hz, and so on².

---

1. The Discrete Fourier Transform (DFT) is a mathematical operation that decomposes a discrete-time signal into its constituent frequencies. The Fast Fourier Transform (FFT) is an efficient algorithm for computing the DFT. Most literature will use these terms interchangeably.

2. One may notice that this means that the FFT bins are linearly spaced. This may raise justified concerns about the pipeline’s ability to resolve the entire frequency spectrum of the piano given its logarithmic nature. The pipeline addresses concern with a “scout” to find the general range of a note and uses specialized DSP techniques to find the fundemental and its harmonics on these cases. More on this later.

The Nyquist Limit and Real Signals

Because audio pipelines process purely real signals (no imaginary component in the time domain), the resulting FFT spectrum exhibits conjugate symmetry around the Nyquist frequency (\frac{f_s}{2}). Mathematically, X[k] = X^*[N-k]. Because the upper half of the bins (from N/2 to N-1) simply mirror the lower half, they contain redundant information. Consequently, standard audio DSP implementations discard the mirrored upper half. If your FFT size is N, the number of usable bins K evaluated in your CSD calculation is typically just the positive frequencies up to Nyquist:

K = \frac{N}{2} + 1

For an N of 2048, your spectrum X actually contains K = 1025 useful frequency bins.

Windowing

Before the pipeline computes the DFT for a given frame to evaluate the CSD, it must first address the mathematical assumptions of the Fourier transform. The DFT inherently assumes that the finite N-sample block repeats infinitely. Unless a frequency perfectly completes an integer number of cycles within that specific block, the artificial edges of the raw audio frame create sharp time-domain discontinuities.

Spectral Leakage

In the frequency domain, a hard time-domain truncation (i.e. operating directly on the buffer) is equivalent to multiplying the continuous signal by a rectangular window. Because multiplication in the time domain equates to convolution in the frequency domain, the signal is convolved with the Fourier transform of a rectangular window: a sinc function (\sin(x)/x).

\begin{align*} \mathcal{F}\{x[n] \cdot w_{\text{rect}}[n]\} &= X[k] * W_{\text{sinc}}[k] \\ &= \sum_{m=0}^{K-1} X[m] \, W_{\text{sinc}}[k-m]\\ &= \sum_{m=0}^{K-1} X[m] \, \frac{\sin(\pi (k-m))}{\pi (k-m)} \end{align*}

This sinc function features high-energy side lobes. Consequently, the energy of a physical frequency that does not perfectly align with a bin center “leaks” heavily across multiple adjacent frequency bins.

[Code to show window of time based signal]

The Impact on Signal Analysis

If the Gatekeeper were to evaluate the FFT without a prior time-domain windowing stage, the mathematical results would be disastrous for transient detection. As a continuous audio signal progresses, the phase relationship at the arbitrary frame boundaries fluctuates randomly from one frame to the next. This shifting discontinuity causes the resulting spectral leakage pattern to be highly volatile. Mathematically, algorithms will misinterpret this fluctuating leakage as a sudden injection of new broadband acoustic energy. This would cause the metric to constantly spike, falsely and repeatedly triggering the nhwrsf_threshold even during a perfectly stable, sustained harmonic note.

The Hann Window Solution

To prevent this mathematical artifact, the pipeline applies a window function (such as a Hann window) to the raw audio frame before performing the FFT. A window function gently tapers the time-domain boundaries of the frame to exactly zero, smoothing out the artificial start and end discontinuities.

The Hann window is defined as:

w[n] = \frac{1}{2}\left(1 - \cos\left(\frac{2\pi n}{N-1}\right)\right)

where n is the sample index and N is the frame size.

The frequency-domain equivalent of a Hann window possesses a slightly wider main lobe but a much faster side-lobe roll-off (18 dB per octave) compared to a rectangular window. This suppresses far-field energy leakage, keeping the frequency representation tightly constrained and ensuring that frame-to-frame spectral differences are caused by actual acoustic changes rather than mathematical boundary errors.

[Code to show window of time based signal]

Characteristic	Rectangular Window (Raw Frame)	Hann Window
Time-Domain Boundaries	Hard truncation (abrupt discontinuity)	Tapered to zero (smooth continuity)
Main Lobe Width	Narrow	Wider
Side Lobe Attenuation	Poor (\sim -13\text{ dB})	Excellent (\sim -31\text{ dB})
Impact on CSD	Erratic (Severe false-positive rate)	Stable (Accurate transient isolation)

This same logic applies to any frequency analysis, not just the spectral flux. Thus, the pipeline and any other frequency analysis pipeline almost always applies a window function to the raw audio frame before performing the FFT.

Memory Management & State Machine Execution (States 1 & 2)

To calculate this without triggering dynamic heap allocations on the hot audio path, the Gatekeeper maintains a pre-allocated 2048-element array (prev_spectrum) to store the history. After calculating the CSD, it performs a zero-allocation copy_from_slice to update this history for the next frame.

The transient detection logic operates in two distinct phases to handle both the detection and the physical decay of the strike:

1. State 1: ATTACK (Detection): The calculated CSD is compared against the csd_transient_threshold (which defaults to 15.0). If the CSD exceeds this threshold, the Gatekeeper declares a hammer strike. It immediately sets the output to SignalState::Unstable, resets any stability counters to zero, and arms a hard delay counter (transient_delay_counter).

2. State 2: TRANSIENT (Decay Delay): Even after the initial instantaneous strike is detected, the physical broadband noise takes time to dissipate into ringing harmonics. To account for this, the Gatekeeper enters a mandatory bypass phase. For as long as the transient_delay_counter is greater than zero, the state machine forcibly evaluates to SignalState::Unstable and decrements the counter. By default, this delay is configured to 10 frames (\sim 464\text{ms} at 44.1kHz), guaranteeing the transient tail has physically died down before harmonic analysis begins.