Difficulty glitch(?)

So I started my career with a Manor on legend Ai. I managed to finish on the podiums a few times and also win one race, so I increased the difficulty to ultimate but I'm not noticing any big difference. It says "ultimate" in all the settings but I don't think it should be possible to qualify 6th with a Manor around Silverstone in the first season and to lead the race for the first stint. Feels like I'm stuck to legend but I'm not really sure. 

Anyone experienced the same thing?