Try switching to a different Encoder selection. The specific encoders available to you will depend on what hardware you have but the one you currently have selected might not be the most suitable for maintaining the game's performance.
If that doesn't improve things, then as Anexide suggested, try reducing the Quality setting.