FGA-SR: Fourier-Guided Attention Upsampling for Image Super-Resolution

Korea University of Technology and Education (KOREATECH)
arXiv 2025
Pre/Post spectrum & feature maps across upsamplers

Comparison of upsampling heads on Urban100 (img_007) with an EDSR backbone (×4 SR). Top row: pre-/post-upsampling feature maps; bottom row: corresponding Fourier magnitudes for Interp+Conv, Deconv, PixelShuffle, and FGA. FGA concentrates energy along true orientations, suppresses folding near Nyquist, and yields cleaner post-upsampling features with fewer checkerboard artifacts.

Abstract

We propose Frequency-Guided Attention (FGA), a lightweight upsampling module for single image super-resolution. Conventional upsamplers, such as Sub-Pixel Convolution, are efficient but frequently fail to reconstruct high-frequency details and introduce aliasing artifacts. FGA addresses these issues by integrating (1) a Fourier feature-based Multi-Layer Perceptron (MLP) for positional frequency encoding, (2) a cross-resolution Correlation Attention Layer for adaptive spatial alignment, and (3) a frequency-domain L1 loss for spectral fidelity supervision. Adding merely 0.3M parameters, FGA consistently enhances performance across five diverse super-resolution backbones in both lightweight and full-capacity scenarios. Experimental results demonstrate average PSNR gains of 0.12–0.14 dB and improved frequency-domain consistency by up to 29%, particularly evident on texture-rich datasets. Visual and spectral evaluations confirm FGA's effectiveness in reducing aliasing and preserving fine details, establishing it as a practical, scalable alternative to traditional upsampling methods.

Method Overview

FGA architecture: FF-MLP + CAL + FL1

FF-MLP. Inject Fourier-encoded (sin/cos) spatial coordinates into an MLP aligned with sub-pixel groups, conditioning high-frequency behavior during PixelShuffle.

CAL (in XRA, Cross-Resolution Attention). Window attention uses HR queries and LR keys/values to correct misalignment and suppress aliasing, with efficient overlapping windows.

FL1 Loss. Direct supervision in the Fourier domain (half-spectrum), constraining both amplitude and phase to reduce ringing/checkerboard artifacts.

Spectral & Feature Analysis

FRC-AUC: Measuring High-Frequency Fidelity

Average FRC curves and equal-pixel ring indexing map

We compute Fourier Ring Correlation (FRC) between the SR output \(\hat{Y}\) and the ground truth \(Y\). The 2D spectrum is partitioned into \(N=64\) equal-pixel rings \(\{\mathcal{R}_i\}_{i=0}^{N-1}\). Using your paper’s formulation, for a spatial frequency radius \(q\) we define:

\[ \mathrm{FRC}(q)= \frac{\displaystyle\sum_{|\mathbf f|=q}F_{1}(\mathbf f)\,F_{2}^{*}(\mathbf f)} {\displaystyle\sqrt{\left(\sum_{|\mathbf f|=q}|F_{1}(\mathbf f)|^{2}\right) \left(\sum_{|\mathbf f|=q}|F_{2}(\mathbf f)|^{2}\right)}} \, . \]

We summarize the high-frequency band by averaging the top quartile of rings. With \(N=64\), the high-frequency index starts at \(i_{\mathrm{HF}}=\lceil 0.75N\rceil = 48\). The FRC-AUC is:

\[ \mathrm{FRC\text{-}AUC} = \frac{1}{N_{\text{HF}}} \sum_{i=i_{\mathrm{HF}}}^{N - 1} \mathrm{FRC}_i, \quad i_{\mathrm{HF}} = \lceil 0.75N \rceil = 48, \quad N_{\text{HF}} = 16 \; (N - i_{\mathrm{HF}}). \]

Top-Quartile Ring–Masked Reconstructions (Rings 48–63)

Only rings 48–63 are retained in the Fourier domain; all other frequencies are zeroed before iFFT.
  Colormap: viridis (purple/blue → yellow = larger magnitude of the band-passed spatial signal).

This visualization is not an FFT magnitude map. We keep only the top-quartile rings (48–63), zero out all other frequencies, and invert back to the spatial domain. FGA suppresses checkerboard-like high-frequency components near the Nyquist region while concentrating energy along true structural directions, yielding spatial patterns closer to the GT.

Empirically, FGA consistently boosts the high-frequency FRC bands across architectures/datasets, indicating closer spectral agreement with GT textures and fewer aliasing artifacts.

FRC-AUC (Top-25% Rings)

Open demo notebook

Quantitative Results

Qualitative Comparisons

Paper

BibTeX

@article{Choi2025FGA,
  title   = {Fourier-Guided Attention Upsampling for Image Super-Resolution},
  author  = {Choi, Daejune and No, Youchan and Lee, Jinhyung and Kim, Duksu},
  journal = {arXiv preprint arXiv:2508.10616},
  year    = {2025}
}