Autonomous vehicles need to accurately forecast future human trajectories in order to navigate safely and gain human trust. Capturing the subtle intricacies of human motion is a challenging task as human trajectories are multimodal i.e. given a past trajectory, multiple socially compliant future trajectories are possible. To this effect, various human trajectory prediction models have been proposed to capture not only the multimodality but also the unspoken social rules of mobility. The current best performing approach is based on Generative Adversarial Network (GAN) taking advantage of the success of Recurrent Neural Network (RNN) models in sequence prediction tasks. In this work, we highlight an unexpected pitfall in this state-of-the-art architecture via controlled experiments. We provide an explanation for this shortcoming and propose a modification to the given architecture leveraging the architectures used in the GAN community. Furthermore, we prove the efficacy of the proposed modification on synthetic data and real world datasets, thereby indicating room for improvement on state-of-the-art.