2 Meg in the Embed, and the little tweet said "Give over"

As the embedded tweet says, I was shocked to discover that a single Twitter embed - inserted into a Wordpress blog using the native embed feature - more than doubled the download size of my site, caused by dozens of Twitter JavaScript resources, all fetched using HTTP/1.1. For tweet that contains just text from me(!), it is unpalatable that so much - well, crap - is pulled in to render it. I’ve come up with a solution now but before that, I sent the above tweet in the hope that someone might have a magic fix. My plan sort of worked, it generated a lot of replies from people more knowledgable about web development than me. The discussion was very interesting, I recommend you read it. But if you don’t have the time for that my summary is:

  1. This is not a “new” problem.
  2. Lots of people assume my website is built on some kind of Server Side Rendering paradigm where I have a build step before deploying static assets.
  3. There are lots of “clever” techniques that can be applied during the build step in order to avoiding the high cost of the Twitter embed code. The meaning of clever in this context seems to vary person-by-person.

Points 2) and 3) were quite eye opening for me. I like to play “dummies advocate” with technology and think about how someone on a managed blog platform might respond to such advice. Could the realistically change their entire CMS over from a nice WYSIWYG editor, to a bunch of technologies with unfamiliar names that need to be installed and managed via continuous integration? While something like Github Pages is nice, elegant and simple for techies like me that use GitHub and Markdown on a daily basis, it doesn’t fly with the family and friends I play tech support for. Imagine how hard it is to simply explain what a repository or a pull request is to someone that has never heard of source control. Some might argue that dynamic sites are too slow in this day and age but read the Background section for why I don’t agree.

I tried to express my point in the Twitter discussion and, to be fair, some pragmatic answers came out. For instance, one suggestion was to just take a screenshot of the tweet and add a hyperlink to Twitter. That actually seems pretty fine, except it would still gall me to encode short text in a bloated image. And it would probably be more work than I could be bothered to do. I hope Twitter might consider offering a “light” embed but I won’t hold my breath.

Towards the end of the replies I got this one from Steren:

This seemed more up my street. Perhaps I could just manually copy the text out of tweets and stick them in an HTML block in my Wordpress editor? Manual work sucks though, surely someone has a script for this right? I Googled for a bit, and started reading into the deep corners of Twitter APIs, backlash over past changes, and the annoyance for the requirements for OAUTH access to do things. That last point in particular seems to be something that affects solutions to the ligher-weight embed challenge, you have to get an API key, and I really can’t be bothered with another thing to worry about.

Solution

During my search, I found Arthur Rump’s page Fallback styling for embedded Tweets. This was awesome because it explains that:

Embedding a Tweet on your website is easy to do. Find the tweet, click on Embed Tweet, copy, paste, done.

Note how all the actual content of the Tweet is just text in a blockquote. That’s great because if the script does not load, the content you wanted to share is still there and readable. This could happen in situations where Twitter is entirely blocked (whether by a company or a nation-state), the user has JavaScript disabled, or because the script is blocked by Content Blocking in Firefox. However, this means that Tweets will be rendered as blockquotes by default.

Arthur Rump

Sure as dammit, after the </blockquote> is <script async src="https://platform.twitter.com/widgets.js". And in my case, I explicitly want to block this thing that loads all the crap! And I want to do this, so that users don’t have to.

So my solution that removes most of the manual steps is a one-liner:

$ curl -s https://publish.twitter.com/oembed?url=[url of tweet] \
| jq -r '.html' | \
sed 's/class="twitter-tweet"/class="twitter-tweet" data-dnt="true"/' \
| sed 's/<script.*<\/script>//g;' | \
tr -d '\n'

Or a simple bash script I like to call embed-tweet-sans-crap.sh:

#! /bin/bash

curl -s https://publish.twitter.com/oembed?url=$1 | jq -r '.html' | sed 's/class="twitter-tweet"/class="twitter-tweet" data-dnt="true"/' | sed 's/<script.*<\/script>//g;' | tr -d '\n'

For example, to grab the tweet that I embedded at the top of this page:

$ ./embed-tweet-sans-crap.sh https://twitter.com/SimmerVigor/status/1345393494559502337

<blockquote class="twitter-tweet" data-dnt="true"><p lang="en" dir="ltr">Due to a single embedded tweet, it seems my website at <a href="https://t.co/jLqWpZKaB6">https://t.co/jLqWpZKaB6</a> loads about 2.5 megabytes of JS across a dozen HTTP/1.1 requests to <a href="https://t.co/7OXJ9hd5oI">https://t.co/7OXJ9hd5oI</a>. WTF, this doubles the total size of download resources. I think I&#39;ll just delete the embed.</p>&mdash; Lucas Pardue (@SimmerVigor) <a href="https://twitter.com/SimmerVigor/status/1345393494559502337?ref_src=twsrc%5Etfw">January 2, 2021</a></blockquote>

Then, in Wordpress embed the tweet by adding a Custom HTML block and pasting the blockquote. As Arthur points out, by default it will look quite plain but it can be fixed using CSS. So I just borrowed Arthur’s and added it as a global custom CSS change. Here’s what it looks like in the editor:

Screenshot of Wordpress WYSIWG editor with HTML block.

Performance Results

So did my work change anything? Here’s a before/after example for the main homepage. I think this is a success even if the embedded tweet is functionally and stylistically basic. It’s better than just deleting the embed…

With crap: ~50 requests, 3.2 MB transferred, 4.4 MB resources.

Sans crap: 31 requests, 453 kB transferred, 956 kB resources.

Note that because of lazy loading, scripts or images don’t appear to have a huge effect on some of the critical events in either case. But it sure doesn’t offend me to see that, due to Cloudflare’s speed features (see Background) DOMContentLoaded and Load times are under 200ms, with Lighthouse reporting a Performance of 96 based on First Contentful Paint 0.4s, Time to Interactive 0.4s, and Largest ContentfulPaint 0.5s. I lost points on Cumulative Layout Shift but I can fix that some other day.

Page load with Twitter embed code. 50 requests, 3.2 MB transferred, 4.4 MB resources

Page load with no crap. 31 requests, 453 kB transferred, 956 kB resources.

Background

My annual end-of-year tradition is to login to my blog and fiddle about. I make grand promises that I’ll do more blogging each year and typically fail. But I also take the opportunity to make some technical change or improvement. This time around I intended to give Cloudflare’s Automatic Platform Optimizer (APO) a spin to see how much it could improve my Wordpress-powered blog. (Disclaimer: I am a Cloudflare employee but I don’t work on this product. My clever colleagues do, and I was keen to see just how turnkey this solution was and its impact on a typical blog, maintained by a lazy owner such as myself would be).

My jist of APO is that it makes your slow, dynamically-rendered, Wordpress site super fast by caching everything in Cloudflare’s edge. It does this via a Wordpress plugin that magically monitors the site and coordinates with Cloudflare to rapidly purge and cache whenever there are changes. Yevgen and Sven’s blog goes into some great detail on the matter https://blog.cloudflare.com/building-automatic-platform-optimization-for-wordpress-using-cloudflare-workers/.

My speciality is on the network protocols side of things. So whenever I’m investigating a website, I’ll fire up the Dev Tools Network panel and start looking at what requests are happening, where they going, how they are performing etc. For deep inspection, I’ll also head over to WebPageTest, but for quick tests local dev normally gives some good indications. The network trace for my site with Twitter embed code was shocking. I wasn’t sure at first where it came from and had to hunt through the blog posts to find it. The embed is a minor decoration and the blog would not have lost anything if I simply removed it. However, I’m happy with the solution I came to especially because it integrates with my existing workflow. I now benefit from loading more content via APO and other speed features on Cloudflare’s edge.