Hi,
Du solltest Dich eher an der Gesamtladezeit der entsprechenden Seiten orientieren. Der TTFB sagt lediglich aus wie lange es dauert, bis der Browser die ersten Daten vom Server erhält. Mit der eigentlichen „User Experience“ hat das eher weniger zu tun.
Um das Problem besser eingrenzen zu können evtl. eine Testumgebung einrichten und der Reihe nach Plugin für Plugin deaktivieren, damit Du den Übeltäter identifizieren kannst.
Hilfreich ist häufig auch eine Waterfall Analyse (z.B. mit Pingdom Speedtest):
Zum Thema TTFB siehe auch:
"Why TTFB doesn’t make sense
TTFB is a widely-used metric because it is easy-to-understand and it is a great signal for connection setup time, server time and network latency. It can help website owners identify when performance issues originate from their server. But is TTFB a good signal for how real users experience the loading speed of a web page in a browser?
When a web page loads in a browser, the user’s perception of speed isn’t related to the moment the browser first receives bytes of data. It is related to when the user starts to see the page rendering on the screen.
The loading of a web page in a browser is a very complex process. Almost all of this process happens after TTFB is reported. After the first byte has been received, the browser still has to load the main HTML file. It also has to load fonts, stylesheets, javascript, images and other resources. Often these resources link to other resources that also must be downloaded. Often these resources entirely block the rendering of the page. Alongside all these downloads, the browser is also parsing the HTML, CSS and JavaScript. It is building data structures that represent the content of the web page as well as how it is styled. All of this is in preparation to start rendering the final page onto the screen for the user."
Viele Grüße