Device and Method of Synthetic Image Generation

Abstract
A computer-implemented method for generating synthetic images using a conditional diffusion model. The method involves providing a neural conditioning, which is determined by a foundation model, as input to a ControlNet. The neural conditioning and a latent input representation are then propagated through the ControlNet, and the outputs of the ControlNet are used as additional injections for the diffusion model. The latent input representation is further propagated through the diffusion model, with the additional injections from the ControlNet being injected into corresponding layers of the diffusion model during propagation.