Swiching Linear Dynamical Systems

Switching Linear Dynamical System

Struct to Encode a Hidden Markov model that switches among K distinct LinearDyanmicalSystems

Fields

• A::V: Transition matrix for mode switching.
• B::VL: Vector of Linear Dynamical System models.
• πₖ::V: Initial state distribution.
• K::Int: Number of modes.

Random.rand(rng, slds, T)

Generate synthetic data with switching LDS models

#Arguments

• rng:AbstractRNG: Random number generator
• slds::SwitchingLinearDynamicalSystem: The switching LDS model
• T::Int: Number of time steps to sample

Returns

• Tuple{Array,Array, Array}: Latent states (x), observations (y), and mode sequences (z).

fit!(slds::SwitchingLinearDynamicalSystem, y::Matrix{T}; 
     max_iter::Int=1000, 
     tol::Real=1e-12, 
     ) where {T<:Real}

Fit a Switching Linear Dynamical System using the variational Expectation-Maximization (EM) algorithm with Kalman smoothing.

Arguments

• slds::SwitchingLinearDynamicalSystem: The Switching Linear Dynamical System to be fitted.
• y::Matrix{T}: Observed data, size (obsdim, Tsteps).

Keyword Arguments

• max_iter::Int=1000: Maximum number of EM iterations.
• tol::Real=1e-12: Convergence tolerance for log-likelihood change.

Returns

• mls::Vector{T}: Vector of log-likelihood values for each iteration.
• param_diff::Vector{T}: Vector of parameter differences over each iteration.
• FB::ForwardBackward: ForwardBackward struct
• FS::FilterSmooth: FilterSmooth struct

hmm_elbo(model::AbstractHMM, FB::ForwardBackward; ϵ::Float64=1e-10)

Compute the evidence based lower bound (ELBO) from the discrete state model.

initialize_slds(;K::Int=2, d::Int=2, p::Int=10, self_bias::Float64=5.0, seed::Int=42)

Initialize a Switching Linear Dynamical System with random parameters.

mstep!(slds::AbstractHMM, FS::Vector{FilterSmooth{T}}, y::AbstractMatrix{T}, FB::ForwardBackward) where {T<:Real}

Function to carry out the M-step in Expectation-Maximization algorithm for SLDS

variational_expectation!(model::SwitchingLinearDynamicalSystem, y, FB, FS) -> Float64

Compute the variational expectation (Evidence Lower Bound, ELBO) for a Switching Linear Dynamical System by executing the following operations:

1. Extract Responsibilities: Retrieves the responsibilities (γ) from the forward-backward object and computes their exponentials (hs).

2. Parallel Smoothing and Sufficient Statistics Calculation: For each regime k from 1 to model.K, the function:

   • Performs smoothing using the smooth function to obtain smoothed states (x_smooth), covariances (p_smooth), inverse off-diagonal terms, and total entropy.
   • Computes sufficient statistics (E_z, E_zz, E_zz_prev) from the smoothed estimates.
   • Calculates the ELBO contribution for the current regime and accumulates it into ml_total.

3. Update Variational Distributions:

   • Computes the variational distributions (qs) from the smoothed states, which are stored as log-likelihoods in FB.
   • Executes the forward and backward passes to update the responsibilities (γ) based on the new qs.
   • Recalculates the responsibilities (γ) to reflect the updated variational distributions.

4. Return ELBO: Returns the accumulated ELBO (ml_total), which quantifies the quality of the variational approximation.

variational_qs!(model::AbstractVector{<:GaussianObservationModel{T, <:AbstractMatrix{T}}}, FB::ForwardBackward, y::AbstractMatrix{T}, FS::Vector{FilterSmooth{T}}) where {T<:Real}

Compute the variational distributions (qs) and update the log-likelihoods for a set of Gaussian observation models within a Forward-Backward framework.