What are Emission Models?

Emission models describe how observations are generated from latent states in a state space model. These models define the conditional distribution of the observed data given the hidden state or input features. In StateSpaceDynamics.jl, a flexible suite of emission models is supported, including both simple parametric distributions and regression-based models.

At a high level, emission models encode:

The distribution of observations (e.g., Gaussian, Poisson, Bernoulli)
How observations relate to inputs or latent states, either directly or via regression

StateSpaceDynamics.EmissionModel — Type

Base type hierarchy for emission models. Each emission model must implement:

sample()
loglikelihood()
fit!()

StateSpaceDynamics.RegressionEmission — Type

Base type hierarchy for regression emission models.

Gaussian Emission Model

The GaussianEmission is a basic model where the observations are drawn from a multivariate normal distribution with a fixed mean and covariance.

\[y_t \sim \mathcal{N}(\mu, \Sigma)\]

This emission model is often used when the observed data is real-valued and homoscedastic.

StateSpaceDynamics.GaussianEmission — Type

mutable struct GaussianEmission <: EmissionModel

GaussianEmission model with mean and covariance.

StateSpaceDynamics.loglikelihood — Method

loglikelihood(model::GaussianEmission, Y::AbstractMatrix{T}) where {T<:Real}

Calculate the log likelihood of the data Y given the Gaussian emission model.

Base.rand — Method

Random.rand(model::GaussianEmission; kwargs...)
Random.rand(rng::AbstractRNG, model::GaussianEmission; n::Int=1)

Generate random samples from a Gaussian emission model.

StateSpaceDynamics.fit! — Method

function fit!(model::GaussianEmission,
        Y::AbstractMatrix{T},
        w::AbstractVector{T}=ones(size(Y, 1))) where {T<:Real}

Fit a GaussianEmission model to the data Y weighted by weights w.

Regression-Based Emission Models

Regression-based emissions allow the output to depend on an input matrix $\Phi$. The regression relationship is defined by a coefficient matrix $\beta$, optionally with an intercept and regularization.

Gaussian Regression Emission

In the GaussianRegressionEmission, the outputs are real-valued and modeled via linear regression with additive Gaussian noise.

\[y_t \sim \mathcal{N}(\Phi_t \beta, \Sigma)\]

StateSpaceDynamics.GaussianRegressionEmission — Type

GaussianRegressionEmission

Store a Gaussian regression Emission model.

Fields

input_dim::Int: Dimension of the input data.
output_dim::Int: Dimension of the output data.
include_intercept::Bool: Whether to include an intercept term; if true, the first column of β is assumed to be the intercept/bias.
β::AbstractMatrix{<:Real} = if include_intercept zeros(input_dim + 1, output_dim) else zeros(input_dim, output_dim) end: Coefficient matrix of the model. Shape inputdim by outputdim. The first row are the intercept terms, if included.
Σ::AbstractMatrix{<:Real}: Covariance matrix of the model.
λ:<Real: Regularization parameter.

Base.rand — Method

Random.rand(model::GaussianRegressionEmission, Φ::Union{Matrix{<:Real},Vector{<:Real}})
Random.rand(
    rng::AbstractRNG,
    model::GaussianRegressionEmission,
    Φ::Union{Matrix{<:Real},Vector{<:Real}}
)

Generate samples from a Gaussian regression model.

StateSpaceDynamics.loglikelihood — Method

loglikelihood(model::GaussianRegressionEmission,
    Φ::AbstractMatrix{T},
    Y::AbstractMatrix{T},
    w::AbstractVector{T}=ones(size(Y, 1))) where {T<:Real}

Calculate the log likelihood of the data Y given the Gaussian regression emission model and the input features Φ.

Bernoulli Regression Emission

The BernoulliRegressionEmission is appropriate for binary data. The probability of success is modeled via a logistic function.

\[p(y_t = 1 \mid \Phi_t) = \sigma(\Phi_t \beta)\]

Where $\sigma(z) = 1 / (1 + e^{-z})$ is the logistic function.

StateSpaceDynamics.BernoulliRegressionEmission — Type

BernoulliRegressionEmission

Store a Bernoulli regression model.

Fields

input_dim::Int: Dimensionality of the input data.
output_dim::Int: Dimensionality of the output data.
β::AbstractMatrix{<:Real}: Bernoulli regression coefficients.
include_intercept::Bool: Whether to include an intercept term.
λ<:Real: L2 Regularization parameter.

```

Base.rand — Method

Random.rand(
    model::BernoulliRegressionEmission,
    Φ::Union{Matrix{<:Real},Vector{<:Real}}
)
Random.rand(
    rng::AbstractRNG,
    model::BernoulliRegressionEmission,
    Φ::Union{Matrix{<:Real},Vector{<:Real}}
)

Generate samples from a Bernoulli regression emission.

StateSpaceDynamics.loglikelihood — Function

function loglikelihood(
    model::BernoulliRegressionEmission,
    Φ::AbstractMatrix{T1},
    Y::AbstractMatrix{T2},
    w::AbstractVector{T3}=ones(size(Y, 1))
) where {T1<:Real, T2<:Real, T3<:Real}

Calculate the log likelihood of the data Y given the Bernoulli regression emission model and the input features Φ. Optionally, a vector of weights w can be provided.

Poisson Regression Emission

The PoissonRegressionEmission is ideal for count data, such as spike counts in neuroscience. It models the intensity of the Poisson distribution as an exponential function of the linear predictors.

\[y_t \sim \text{Poisson}(\lambda_t), \quad \lambda_t = \exp(\Phi_t \beta)\]

StateSpaceDynamics.PoissonRegressionEmission — Type

PoissonRegressionEmission

A Poisson regression model.

Fields

input_dim::Int: Dimensionality of the input data.
output_dim::Int: Dimensionality of the output data.
β::AbstractMatrix{<:Real}: The regression coefficients matrix.
include_intercept::Bool: Whether to include a regression intercept.
λ::Real;: L2 Regularization parameter.

Base.rand — Method

Random.rand(
    model::PoissonRegressionEmission,
    Φ::Union{Matrix{<:Real},Vector{<:Real}}
)
Random.rand(
    rng::AbstractRNG,
    model::PoissonRegressionEmission,
    Φ::Union{Matrix{<:Real},Vector{<:Real}}
)

Generate samples from a Poisson regression emission model.

StateSpaceDynamics.loglikelihood — Function

loglikelihood(
    model::PoissonRegressionEmission,
    Φ::AbstractMatrix{T1},
    Y::AbstractMatrix{T2},
    w::AbstractVector{T3}=ones(size(Y, 1))
) where {T1<:Real, T2<:Real, T3<:Real}

Calculate the log-likelihood of a Poisson regression model.

Autoregressive Emission Models

The AutoRegressionEmission models the observation at time t as depending on previous observations (i.e., an autoregressive structure), using a wrapped GaussianRegressionEmission.

\[y_t \sim \mathcal{N}(\sum_{i=1}^p A_i y_{t-i}, \Sigma)\]

Where p is the autoregressive order and A_i are regression weights.

This model is useful when modeling temporal dependencies in the emission process, independent of latent dynamics.

StateSpaceDynamics.AutoRegressionEmission — Type

AutoRegressionEmission <: EmissionModel

Store an autoregressive emission model, which wraps around a GaussianRegressionEmission.

Fields

output_dim::Int: The dimensionality of the output data
order::Int: The order of the Autoregressive process
innerGaussianRegression::GaussianRegressionEmission: The underlying Gaussian regression model used for the emissions.

Base.rand — Method

Random.rand(model::AutoRegressionEmission, X::Matrix{<:Real})
Random.rand(rng::AbstractRNG, model::AutoRegressionEmission, X::Matrix{<:Real})

Generate samples from an autoregressive emission model.

StateSpaceDynamics.loglikelihood — Method

loglikelihood(
    model::AutoRegressionEmission,
    X::AbstractMatrix{T},
    Y::AbstractMatrix{T},
    w::Vector{T}=ones(size(Y, 1))
) where {T<:Real}

Calculate the log likelihood of the data Y given the autoregressive emission model and the previous observations X.

Fitting Regression Emission Models

All regression-based emissions can be fitted using maximum likelihood with optional weights and L2 regularization. Internally, StateSpaceDynamics.jl formulates this as an optimization problem, solved using gradient-based methods (e.g., LBFGS).

StateSpaceDynamics.fit! — Method

fit!(
    model::RegressionEmission,
    X::AbstractMatrix{T1},
    y::AbstractMatrix{T2},
    w::AbstractVector{T3}=ones(size(y, 1))
) where {T1<:Real, T2<:Real, T3<:Real}

Fit a regression emission model give input data X, output data y, and weights w.

Arguments

- `model::RegressionEmission`: A regression emission model.
- `X::AbstractMatrix{<:Real}:`: Input data.
- `y::AbstractMatrix{<:Real}`: Output data.
- `w::AbstractVector{<:Real}`: Weights to define each point's contribution to the fit.

Returns

- `model::RegressionEmission`: The regression model with the newly updated parameters.