As a mobile user changes computing environment, his context and available resources change. In a ubiquitous multimodal multimedia computing system, the userís context determines the appropriate modalities of man machine interaction. The available resources, however, determine the media for activation to support the chosen modalities. This paper presents a paradigm of such computing system that exhibits robustness in context adaptation. Technical challenges in formulating this systemís infrastructure include: (1) the definition of relationship between context and suitable modalities, and the quantification of such suitability, (2) the classification of media and its relationship to modality, (3) the development of tool for incremental definition of context parameters, (4) the definition of mechanism for the systemís fault-tolerance to a failed or missing media device, and (5) a means of systemís adaptation to a newly-introduced media device. The heart of this paradigmís design is the machine learningís knowledge acquisition and the use of the layered virtual machine for incremental user context. Data validation is presented through formal specification and scenario simulations.