Dual Cone Projectors

class torchjd.linalg.DualConeProjector

Abstract class whose instances are responsible for projecting vectors onto the dual cone of the rows of a matrix, or rather the dual form of this problem.

abstractmethod __call__(U, G)

Computes for each vector \(u\) in the provided tensor U the weights \(w\) of the projection of \(J^\top u\) onto the dual cone of the rows of \(J\), provided \(G = J J^\top\) and \(u\). In other words, this computes the \(w\) that satisfies \(\pi_J(J^\top u) = J^\top w\), with \(\pi_J\) defined in Equation 3 of [1].

By Proposition 1 of [1], this is equivalent to solving for \(v\) the following quadratic program:

\[\begin{split}\min_{v} \quad & v^\top G v \\ \text{subject to} \quad & u \preceq v\end{split}\]

Reference: [1] Jacobian Descent For Multi-Objective Optimization.

Parameters:

• U (Tensor) – The tensor of weights corresponding to the vectors to project, of shape [..., m].

• G (PSDMatrix) – The Gramian matrix of shape [m, m]. It must be symmetric and positive semi-definite.

Return type:

Tensor

Returns:

A tensor of projection weights with the same shape as U.

class torchjd.linalg.QuadprogProjector(*, norm_eps=0.0001, reg_eps=0.0001)

Solves the quadratic program defined in DualConeProjector.__call__() using the quadprog QP solver.

Parameters:

• norm_eps (float) – A small value to avoid division by zero when normalizing.

• reg_eps (float) – A small value to add to the diagonal of the gramian of the matrix. Due to numerical errors when computing the gramian, it might not exactly be positive definite. This issue can make the optimization fail. Adding reg_eps to the diagonal of the gramian ensures that it is positive definite.