_fir_filter.py
Self-contained FIR filter design and out-of-core FIR filtering.
Vendored from ghostipy (Apache-2.0, https://github.com/kemerelab/ghostipy),
covering the FIR subset the spyglass LFP pipeline needs. A copy of the Apache
License 2.0 ships with spyglass at
spyglass/common/licenses/ghostipy-Apache-2.0.txt; the modifications made
here are listed under "Intentional divergences from upstream" below.
Vendored from upstream (upstream name -> name here):
estimate_taps->estimate_taps, FIR tap-count estimatefirdesign->firdesign, Type I design with spline transition bands (L2), and its_firsplinelow-pass prototype helpergroup_delay->group_delay, integer group delay of a Type I FIRfilter_data_fir->filter_data_fir, themode='full'entry pointosconvolve->_osconvolve, the overlap-save engine beneath it, which streams into a (possibly on-disk) preallocated output array with decimation and per-dimension index restrictions. Its argument validation and output-shape planning were split out into_plan_osconvolve, which has no upstream counterpart.
common_filter.py calls estimate_taps, firdesign and
filter_data_fir; group_delay is vendored and exported for completeness
but spyglass uses its own FirFilterParameters.calc_filter_delay instead.
Call signatures keep upstream ghostipy's parameter order and defaults, with
clearer parameter names (e.g. fs -> sampling_freq, tw ->
transition_width, p -> spline_power, b -> filter_coeffs)
and added type annotations; numeric output matches upstream to floating-point
round-off (measured: firdesign coefficients bitwise identical, filtered
output within 4.4e-16 on spyglass's LFP calls).
Intentional divergences from upstream:
- FFT backend:
pyfftwreplaced withscipy.fft(workers=for multithreading), using the real transform (rfft/irfft) for real input and the full complex transform otherwise. This removed the M1-Mac / conda-forge install friction the spyglass setup notes used to document, and is the only change that touches the numeric path for valid input (round-off only). - Block selection at boundaries: when the requested (exclusive) output stop lands exactly on an overlap-save block boundary, the trailing empty block is no longer processed. Upstream still read and FFT'd it only to write zero samples; skipping it removes a wasted transform and a read one block past the needed data (which could fail on a strict lazy/on-disk signal). Output is identical.
- Overlap placement bug fix: upstream positioned a block's leading M-1 overlap
samples from the LENGTH of the (clipped) read, which is only correct when the
read reaches the block start. For a signal shorter than M-1 filtered with an
nffttight enough to need several blocks, the read is clipped at both ends and upstream shifted those samples, returning a wrong convolution (e.g.[2, 4, 4, 4, 10]instead of[2, 4, 6, 8, 10]). The position is now derived from where the read actually starts. Unreachable at spyglass's defaultnfft(>= 10x the kernel), so LFP output is unaffected. - Fail-loud / fail-closed hardening that affects only invalid inputs or genuine
errors (never the valid path spyglass exercises): the M-1 overlap read no
longer swallows exceptions and silently zero-fills;
input_index_bounds/output_index_boundstreat the stop as exclusive and validate by range rather than probing the array;estimate_tapsrejects non-positivesampling_freq,transition_width,passband_deviation, andstopband_deviation, and deviations so loose the tap estimate would be < 1;firdesign/_firsplinerequire an integernumtaps>= 1 and at least two orderedband_edges; the spline powerspline_powermust be0;
decimation_factormust be an integer >= 1;nfftmust be an integer >= the kernel length;input_dim_restrictionsentries must be 1-D integer index arrays restricting at most one non-filtered axis;output_offsetmust be an integer >= 0 that fits withinoutarray; complex input no longer raisesUnboundLocalError. - Provably empty block reads are skipped rather than issued: the leading overlap
read when a block starts at sample 0 (nothing precedes it), and the main read
when a trailing block starts at or past the end of the data. Upstream issued
both unconditionally and hid the fallout in the blanket
exceptabove. h5py rejects an empty slice combined with a fancy index of 16 or more elements ("Dataspaces don't have hyperslab selections"), which is an ordinary LFP configuration. The block buffer is already zeroed, so skipping is also the correct fill. input_dim_restrictionsmay select in any order, including duplicates, and rows are returned in the order requested. An unsorted selection is READ as sorted unique indices -- all h5py accepts -- then gathered back. Upstream passed the array straight through, which worked only for a sorted selection on an on-disk signal.outarray's real/complex check reads itsdtypeinstead of a slice of its contents, so no data is read from a (possibly on-disk) output array.verboseis gone from both entry points; progress goes to a module-levellogginglogger instead ofprint.
Design details (spline-transition FIR) follow Burrus et al., 1992.
group_delay(filter_coeffs)
¶
Group delay of a linear-phase (Type I) FIR filter.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filter_coeffs
|
(ndarray, shape(N))
|
The filter coefficients. |
required |
Returns:
| Type | Description |
|---|---|
int
|
The group delay in samples, |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Source code in src/spyglass/common/_fir_filter.py
estimate_taps(sampling_freq, transition_width, *, passband_deviation=0.001, stopband_deviation=1e-06)
¶
Estimate the number of taps for a Type I FIR filter.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
sampling_freq
|
float
|
Sampling rate in Hz. |
required |
transition_width
|
float
|
Transition bandwidth in Hz. |
required |
passband_deviation
|
float
|
Passband deviation. Default is 0.1% (1e-3). |
0.001
|
stopband_deviation
|
float
|
Minimum stopband attenuation. Default is 120 dB (1e-6). |
1e-06
|
Returns:
| Type | Description |
|---|---|
int
|
Number of taps (always odd). |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
References
https://dsp.stackexchange.com/questions/31066
Source code in src/spyglass/common/_fir_filter.py
firdesign(numtaps, band_edges, desired, *, sampling_freq=1, spline_power=None)
¶
Design an arbitrary Type I FIR filter with spline transition bands.
Optimized for an L2 error norm.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
numtaps
|
int
|
Number of filter coefficients (must be a positive odd integer). |
required |
band_edges
|
(array_like, shape(2 * n_bands))
|
Critical frequencies of the filter in Hz, an even-length, strictly increasing sequence. Do not include 0 or the Nyquist frequency. |
required |
desired
|
(array_like, shape(2 * n_bands))
|
Magnitude response at each band edge; each value must be 0 or 1. The values must alternate between transition bands (the two edges of a transition band differ) and flat bands (the two edges match). |
required |
sampling_freq
|
float
|
Sampling rate in Hz. Default is 1 Hz. |
1
|
spline_power
|
float
|
Power for the spline transition-band functions. Default follows Burrus et al., 1992. |
None
|
Returns:
| Type | Description |
|---|---|
(ndarray, shape(numtaps))
|
The filter coefficients. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Source code in src/spyglass/common/_fir_filter.py
343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 | |
describe_output(data, filter_coeffs, *, nfft=None, axis=-1, input_index_bounds=None, output_index_bounds=None, decimation_factor=None, input_dim_restrictions=None)
¶
Shape and dtype :func:filter_data_fir would produce, without filtering.
Sizes the preallocated (possibly on-disk) array for the out-of-core
streaming protocol described in :func:filter_data_fir. No data is read and
no FFT runs -- this only validates the arguments and plans the output.
Every argument that affects the answer is accepted and validated exactly as
:func:filter_data_fir validates it, so whatever this accepts, the matching
filtering call accepts too. The arguments that cannot affect the answer
(threads, outarray, output_offset) are deliberately absent
rather than accepted and ignored.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
See
|
|
required |
Returns:
| Name | Type | Description |
|---|---|---|
shape |
tuple of int
|
Shape the output would have, including the effect of
|
dtype |
str
|
|
Raises:
| Type | Description |
|---|---|
(ValueError, IndexError)
|
On the same invalid arguments :func: |
Source code in src/spyglass/common/_fir_filter.py
filter_data_fir(data, filter_coeffs, *, nfft=None, threads=cpu_count(), axis=-1, outarray=None, input_index_bounds=None, output_index_bounds=None, decimation_factor=None, input_dim_restrictions=None, output_offset=0)
¶
Apply an FIR filter to data via overlap-save FFT convolution.
This is the public entry point spyglass uses: a thin mode='full' wrapper
over the general :func:_osconvolve engine, exposing only the parameters
spyglass needs. Combined with output_index_bounds set to
[group_delay, group_delay + N] the full-mode convolution yields the
zero-phase, delay-compensated output that spyglass relies on.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
ndarray or Dataset
|
The data to be filtered, shape |
required |
filter_coeffs
|
(array_like, shape(M))
|
Filter coefficients (1-D). Converted to a NumPy array internally. |
required |
nfft
|
int
|
FFT length along the filtered axis; must be an integer >= |
None
|
threads
|
int
|
Number of FFT worker threads (>= 1). Default is the CPU count. |
cpu_count()
|
axis
|
int
|
Axis along which to filter. Default is -1. |
-1
|
outarray
|
ndarray or Dataset
|
Preallocated output (may be on disk; spyglass writes into an NWB dataset). Default allocates in memory. See Notes for the dtype contract. |
None
|
input_index_bounds
|
sequence of 2 int
|
|
None
|
output_index_bounds
|
sequence of 2 int
|
|
None
|
decimation_factor
|
int
|
Integer decimation factor (>= 1). Default None (no decimation). |
None
|
input_dim_restrictions
|
sequence
|
One entry per dimension of |
None
|
output_offset
|
int
|
Offset (>= 0) into |
0
|
Returns:
| Type | Description |
|---|---|
numpy.ndarray or the ``outarray`` type
|
The filtered (and optionally decimated) data. When an |
Raises:
| Type | Description |
|---|---|
ValueError
|
On invalid arguments -- e.g. |
IndexError
|
If |
TypeError
|
If the result is complex but a real-dtype |
Notes
Output dtype: real input yields float64 ('<f8'), complex input
yields complex128 ('<c16'). If you supply your own outarray, its
dtype is used as-is and the result is cast into it -- assigning the float
result into an integer array truncates silently, so match the dtype from
:func:describe_output (a lower-precision float such as float32 is
fine).
Out-of-core streaming protocol (how spyglass filters data larger than RAM):
call :func:describe_output once per interval to get each interval's output
length, preallocate a single (possibly on-disk) array sized to their sum,
then call this function per interval with that array as outarray and the
running cumulative length as output_offset.
The input is assumed finite: a NaN/inf in any block spreads across that whole block's output via the FFT.
Source code in src/spyglass/common/_fir_filter.py
1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 | |