Learning world models in Model-Based Reinforcement Learning (MBRL) facilitates sample-efficient learning while mitigating model-bias by employing uncertainty estimates in the transition dynamics. This approach also enables the utilization of Model Predictive Control (MPC) and Recurrent Neural Networks (RNN) for effective action planning. However, the current methodologies do not offer uncertainty estimates for the prediction (transition) step in a closed form, which could be instrumental for analytically estimating changes in the dynamics function. In this paper, we introduce a hybrid approach designed to capture both aleatoric (data) uncertainty through Probabilistic Deep Neural Networks (PrDNN) and epistemic (model) uncertainty via a Bayesian Last Layer. By adopting an analytical approach, we can identify alterations in dynamics necessitating the learning and memorization of diverse world models. We present a memory schema and demonstrate its efficacy in swiftly adapting to changes in dynamic functions by retrieving previously learned models.