GeneralizedRandomFourierFeatures.jl

BetaExponentialDistribution(β, γ)

Distribution for R = -log(X) where X ~ Beta(β, γ). Used as the spectral mixing distribution for Beta kernel.

BetaKernel(α, β)

Beta Kernel via spectral Beta mixture:

\[k(r) = B(α, β)\,U\bigl(α,\,1-β,\,(r/ℓ)^2\bigr)\]

where U is Tricomi's confluent hypergeometric function.

GammaExponentialKernel(; γ, metric=KernelFunctions.Euclidean())

Gamma Exponential (Exponential Power) Kernel.

Definition

For inputs $x, x'$ and exponent parameter $\gamma \in (0,2]$, the kernel is defined as:

\[k(x, x') = \exp(-\|x - x'\|^{\gamma})\]

This is also known as Exponential Power, Generalized Gaussian, or Subbotin Kernel.

Parameters for Generalized RFF

• R = 1 (constant)
• λ = 1
• α = γ

Special Cases

• γ = 2: Reduces to the Gaussian (RBF) kernel
• γ = 1: Reduces to the Laplacian kernel

Note

This kernel is positive definite for 0 < γ ≤ 2.

GeneralizedCauchyKernel(α, β)

Generalized Cauchy kernel.

Defenition

For inputs $x, x'$, exponent parameter $\alpha\in(0,2]$ and $\beta>0$, the generalized Cauchy kernel is defined as

\[k(x, x') = \frac{1}{\bigl(1 + \|x - x'\|^{\alpha} / 2\beta \bigr)^{\beta}\]

parameters for generalized RFF approximation: (R ~ Gamma(β, 1), λ = 1/(2β), α) When $β = 1$, it reduces to the Cauchy kernel. When $β → ∞$, it approaches the Exponential Power Kernel.

Kummer Kernel via spectral F-distribution mixture: k(r) = U(α, β, (r/ℓ)^2) where U is confluent hypergeometric U function

RFFBasis{Tinner,Touter,Tω,Tτ,Tsample}

Random Fourier Feature basis for kernel approximation.

Stores sampled frequencies ω and phases τ, and computes the feature map ϕ(x) = outer_weights * cos(ω' * (x / inner_weights) + τ).

Fields

• inner_weights: Lengthscale parameter (divides input)
• outer_weights: Output scaling (typically √(2/M))
• ω: Sampled frequencies of size (input_dims, num_features)
• τ: Sampled phases of size (num_features,)
• sample_params: Callable that returns new (ω, τ) samples

Tricomi Kernel via Beta-exponential mixture: k(r) = U(α, α - β + 1, (r/ℓ)^2) where U is Tricomi's confluent hypergeometric function

build_grff_weight_space_approx(rng::AbstractRNG, input_dims::Integer, num_features::Integer)

Builds a closure grff_weight_space_approx(f::AbstractGP) which takes an AbstractGP as input and constructs a Bayesian linear regression model which approximates f. f is assumed to be a zero mean prior GP with one of the kernels supported by this package.

resample!(ϕ::RFFBasis)

Resample frequencies and phases in-place.

rff_kernelmatrix(basis::RFFBasis, X::AbstractVector)

Compute approximate kernel matrix using Random Fourier Features.

Given a collection of data points X and an RFF basis basis, computes the approximate kernel matrix K̂ where K̂[i,j] ≈ k(X[i], X[j]).

Arguments

• basis: RFF basis generated by sample_generalized_rff_basis
• X: Vector of data points (each point is a vector)

Returns

• K: Approximate kernel matrix (N × N) where N = length(X)

Example

using GeneralizedRandomFourierFeatures, KernelFunctions

# Create kernel and sample RFF basis
k = GeneralizedCauchyKernel(1.5, 2.0)
basis = sample_generalized_rff_basis(k, 3, 200)

# Generate data
X = [rand(3) for _ in 1:50]

# Compute approximate kernel matrix
K_approx = rff_kernelmatrix(basis, X)

# Compare with exact kernel matrix
K_exact = kernelmatrix(k, X)

rff_kernelmatrix(rng::AbstractRNG, kernel::Kernel, X::AbstractVector, M::Int)

Convenience function to sample RFF basis and compute kernel matrix in one call.

Arguments

• rng: Random number generator
• kernel: Kernel function
• X: Vector of data points
• M: Number of random features

Returns

• K: Approximate kernel matrix (N × N)

Example

using GeneralizedRandomFourierFeatures, Random

rng = MersenneTwister(1234)
k = GeneralizedCauchyKernel(1.5, 2.0)
X = [rand(rng, 3) for _ in 1:50]
K = rff_kernelmatrix(rng, k, X, 200)

sample_Sα(rng::AbstractRNG, α::Float64, d::Int)

Sample a d-dimensional symmetric alpha-stable vector Sα using the Gaussian scale mixture representation from Proposition 1.

Mathematical Background

For α ∈ (0, 2], the symmetric α-stable random vector Sα can be represented as: Sα = √(2Aα) · N where N ~ MvNormal(0, I) and Aα is a random scale factor.

Special Cases

• α = 2: Sα = √2 · N (Gaussian case, A₂ = 1)
• α = 1: Sα follows a multivariate Cauchy distribution

Arguments

• rng: Random number generator
• α: Stability parameter (0 < α ≤ 2)
• d: Dimension of the output vector

Returns

• Sα: d-dimensional symmetric α-stable random vector

Reference

Langrené, Warin & Gruet (2024), arXiv:2411.02770v3, Proposition 1

Sample an RFF basis for any supported isotropic kernel.

Arguments

• rng: RNG instance (e.g., MersenneTwister)
• k: A KernelFunctions.Kernel subtype (supports RBF and extended kernels)
• input_dims: Dimensionality of input data
• num_features: Number of random features M

Returns

An RFFBasis object representing the random feature map.

Spectral mixing distribution R for a Beta kernel: R ∼ Beta-exponential(β, γ) where -log(R) follows a Beta distribution with parameters β and γ.

Spectral mixing distribution R for a generalized Cauchy kernel R ∼ Gamma(shape=β, scale=1)

Spectral mixing distribution R for a gamma exponential kernel: R = Constant 1

Spectral mixing distribution R for a generalized Mate`rn kernel: R ∼ InverseGamma(ν, 1.)

Spectral mixing distribution R for a Kummer kernel R ∼ Beta(β, γ)

Spectral mixing distribution R for a Tricomi kernel R ∼ F(2β, 2γ)

Spectral parameters (α, λ=1) for a Beta kernel

Spectral parameters (α, λ=1/2β) for a generalized Cauchy kernel

Spectral parameters (α, λ=1) for a gamma exponential kernel

Spectral parameters (α, λ=ν/2) for a generalized Mate`rn kernel

Spectral parameters (α, λ=1) for a Kummer kernel

Spectral parameters (α, λ=1) for a Tricomi kernel