This paper includes a marked Hawkes process in the original HJM set-up, and investigates the impact of this assumption on the pricing of the popular vanilla fixed-income derivatives. Our model exhibits a smile that can fit the implied volatility of swaptions for a given key rate (tenor). We harness on the log-normality of the model, conditionally with respect to jumps, and derive formulae to evaluate both caplets/floorlets and swaptions. Our model exhibits negative jumps on the zero-coupon (hence positive on the rates). Therefore, its behaviour is compatible with the situation where globally low interest rates can suddenly show cluster of positive jumps in case of tensions on the market. One of the main difficulties when dealing with the HJM model is to keep a framework that is Markovian. In particular, it is important to preserve the important features of the Hull and White version, especially the reconstruction formula that provides the zero-coupon bonds in terms of the underlying model factors. In our case, this formula is based on two factors: a classical Gaussian part and a pure jump martingale part based on a Hawkes process.