Swiching Linear Dynamical Systems

Switching Linear Dynamical System

fit!(slds::SwitchingLinearDynamicalSystem, y::Matrix{T}; 
     max_iter::Int=1000, 
     tol::Real=1e-12, 
     ) where {T<:Real}

Fit a Switching Linear Dynamical System using the variational Expectation-Maximization (EM) algorithm with Kalman smoothing.

Arguments

• slds::SwitchingLinearDynamicalSystem: The Switching Linear Dynamical System to be fitted.
• y::Matrix{T}: Observed data, size (obsdim, Tsteps).

Keyword Arguments

• max_iter::Int=1000: Maximum number of EM iterations.
• tol::Real=1e-12: Convergence tolerance for log-likelihood change.

Returns

• mls::Vector{T}: Vector of log-likelihood values for each iteration.

Initialize a Switching Linear Dynamical System with random parameters.

Generate synthetic data with switching LDS models

variational_expectation!(model::SwitchingLinearDynamicalSystem, y, FB, FS) -> Float64

Compute the variational expectation (Evidence Lower Bound, ELBO) for a Switching Linear Dynamical System.

Arguments

• model::SwitchingLinearDynamicalSystem: The switching linear dynamical system model containing parameters such as the number of regimes (K), system matrices (B), and observation models.

• y: The observation data, typically a matrix where each column represents an observation at a specific time step.

• FB: The forward-backward object that holds variables related to the forward and backward passes, including responsibilities (γ).

• FS: An array of FilterSmooth objects, one for each regime, storing smoothed state estimates and covariances.

Returns

• Float64: The total Evidence Lower Bound (ELBO) computed over all regimes and observations.

Description

This function performs the variational expectation step for a Switching Linear Dynamical System by executing the following operations:

1. Extract Responsibilities: Retrieves the responsibilities (γ) from the forward-backward object and computes their exponentials (hs).

2. Parallel Smoothing and Sufficient Statistics Calculation: For each regime k from 1 to model.K, the function:

   • Performs smoothing using the smooth function to obtain smoothed states (x_smooth), covariances (p_smooth), inverse off-diagonal terms, and total entropy.
   • Computes sufficient statistics (E_z, E_zz, E_zz_prev) from the smoothed estimates.
   • Calculates the ELBO contribution for the current regime and accumulates it into ml_total.

3. Update Variational Distributions:

   • Computes the variational distributions (qs) from the smoothed states, which are stored as log-likelihoods in FB.
   • Executes the forward and backward passes to update the responsibilities (γ) based on the new qs.
   • Recalculates the responsibilities (γ) to reflect the updated variational distributions.

4. Return ELBO: Returns the accumulated ELBO (ml_total), which quantifies the quality of the variational approximation.

Example

```julia

Assume model is an instance of SwitchingLinearDynamicalSystem with K regimes

y is the observation matrix of size (numfeatures, numtime_steps)

FB is a pre-initialized ForwardBackward object

FS is an array of FilterSmooth objects, one for each regime

elbo = variational_expectation!(model, y, FB, FS) println("Computed ELBO: ", elbo)

variational_qs!(model::Vector{GaussianObservationModel{T}}, FB, y, FS) where {T<:Real}

Compute the variational distributions (qs) and update the log-likelihoods for a set of Gaussian observation models within a Forward-Backward framework.

Arguments

• model::Vector{GaussianObservationModel{T}} A vector of Gaussian observation models, where each model defines the parameters for a specific regime or state in a Switching Linear Dynamical System. Each GaussianObservationModel should contain fields such as the observation matrix C and the observation noise covariance R.

• FB The Forward-Backward object that holds variables related to the forward and backward passes of the algorithm. It must contain a mutable field loglikelihoods, which is a matrix where each entry loglikelihoods[k, t] corresponds to the log-likelihood of the observation at time t under regime k.

• y The observation data matrix, where each column represents an observation vector at a specific time step. The dimensions are typically (num_features, num_time_steps).

• FS An array of FilterSmooth objects, one for each regime, that store smoothed state estimates (x_smooth) and their covariances (p_smooth). These are used to compute the expected sufficient statistics needed for updating the variational distributions.

Returns

• Nothing The function performs in-place updates on the FB.loglikelihoods matrix. It does not return any value.

Description

variational_qs! updates the log-likelihoods for each Gaussian observation model across all time steps based on the current smoothed state estimates. This is a critical step in variational inference algorithms for Switching Linear Dynamical Systems, where the goal is to approximate the posterior distributions over latent variables.

Example

```julia

Define Gaussian observation models for each regime

model = [ GaussianObservationModel(C = randn(5, 10), R = Matrix{Float64}(I, 5, 5)), GaussianObservationModel(C = randn(5, 10), R = Matrix{Float64}(I, 5, 5)) ]

Initialize ForwardBackward object with a preallocated loglikelihoods matrix

FB = ForwardBackward(loglikelihoods = zeros(Float64, length(model), 100))

Generate synthetic observation data (5 features, 100 time steps)

y = randn(5, 100)

Initialize FilterSmooth objects for each regime

FS = [ initialize_FilterSmooth(model[k], size(y, 2)) for k in 1:length(model) ]

Compute variational distributions and update log-likelihoods

variational_qs!(model, FB, y, FS)

Access the updated log-likelihoods

println(FB.loglikelihoods)