We propose a general partially-observed framework of Markov processes with marked point process observations for ultra-high frequency (UHF) transaction price data, allowing other observable economic or market factors. We develop the corresponding Bayesian inference via filtering equations to quantify parameter and model uncertainty. Specifically, we derive filtering equations to characterize the evolution of the statistical foundation such as likelihoods, posteriors, Bayes factors and posterior model probabilities. Given the computational challenge, we provide a convergence theorem, enabling us to employ the Markov chain approximation method to construct consistent, easily-parallelizable, recursive algorithms. The algorithms calculate the fundamental statistical characteristics and are capable of implementing the Bayesian inference in real-time for streaming UHF data, via parallel computing for sophisticated models. The general theory is illustrated by specific models built for U.S. Treasury Notes transactions data from GovPX and by Heston stochastic volatility model for stock transactions data. This is joint work with B. Bundick, X. Hu, and D. Kuipers.