Joost.blog

How I made my skills update themselves

joost@joost.blog (Joost de Valk) — Tue, 14 Apr 2026 00:00:00 GMT

I updated one of my Agent Skills today and realized I had no way to tell my other machines they were running a stale copy. Skills install as loose folders in ~/.claude/skills/ (or wherever your agent puts them). There's no npm outdated, no brew upgrade, no update daemon. The skill just runs whatever's on disk.

So I added a small pattern that makes each skill check itself on invocation. If it's out of date, the skill offers to install the update before continuing. I couldn't find anyone doing quite this. The whole thing is a few lines of configuration per skill.

Why versioning matters when posts and skills ship together

Most of my skills are the executable companion to a post:

The astro-seo skill implements Astro SEO: the definitive guide.
The github-repo skill implements How to create a healthy GitHub repository.
The github-profile skill implements Good-looking GitHub profile pages.
The wp-github-actions skill implements GitHub Actions to keep your WordPress plugin healthy.

Each pairing turns an opinionated blog post into a drop-in workflow.

This is a shipping pattern I like a lot: the post argues for a set of choices, the skill makes those choices the default. Want to understand why? Read the post. Want to apply it? Run the skill.

But opinions evolve. When I update the Astro SEO guide to cover a new build-time validator, I update the skill in the same pass. If users don't know their installed skill is behind the post, they'll follow instructions that no longer match what I'd write today. An Agent Skill without a version check is cached documentation. Useful until it's wrong. And you don't find out until it is.

That's the specific problem I'm solving. The pattern below is general, but this is why it matters to me.

The pattern

Four pieces.

First, every SKILL.md carries a version: field in its frontmatter:

---
name: astro-seo
version: "0.4"
description: >
  Audits and improves SEO for Astro sites...
---

Second, a single versions.json at the repo root maps each skill name to its current version:

{
    "astro-seo": "0.4",
    "readability-check": "0.4",
    "github-repo": "0.3"
}

Third, every SKILL.md includes a short paragraph that tells the skill to self-check when it runs — and, if the user approves, to install the update inline:

Before running, fetch https://raw.githubusercontent.com/jdevalk/skills/main/versions.json
and compare the `astro-seo` entry to the `version:` in this file's frontmatter.
If the manifest version is higher, tell the user the skill is out of date and
offer to update it now. If they agree, run:

    curl -fsSL https://github.com/jdevalk/skills/releases/latest/download/astro-seo.skill \
      -o /tmp/astro-seo.skill \
      && unzip -oq /tmp/astro-seo.skill -d <parent of this skill's directory> \
      && rm /tmp/astro-seo.skill

After the unzip, ask the user to re-invoke the skill so the new version loads
into context. The check is informational and never blocks: if the user
declines, continue with the rest of the workflow on the current version.

Fourth, a CI job that fails any PR where a skill's frontmatter version doesn't match its versions.json entry. The manifest and the shipped skills can't drift.

That's it. No runtime. No service. The skill checks on each invocation using the agent's own WebFetch tool. If the user approves the update, the skill re-uses the Bash tool that's already there to pull the latest release and unpack it in place. Users find out they're behind — and can be caught up — at exactly the moment it matters: when they're about to run the skill.

Why no cache

The obvious concern is that every skill invocation now costs a network round-trip. Three reasons I left it alone:

The WebFetch tool already has a built-in 15-minute cache. Multiple skill invocations in one work session share the response, which covers the realistic hot path. I'm not going to beat that with hand-rolled caching.

A longer cache fights the purpose. The whole point of the check is to alert me about updates. A 24-hour cache means I could miss a release for a day. The check is already non-blocking. A cache miss costs less than running a stale skill for another day.

The fetch is around a hundred milliseconds against a static GitHub raw URL. In a human-paced workflow where I invoke a skill once or twice per session, it's noise. If I ever ship fifty skills and it matters, the better fix is to centralize. One shared manifest fetch per session, not one per skill. But that needs harness support Claude Code doesn't expose to skill authors today. So the skill is the smallest unit with an execution context, and that's where the check lives.

What I couldn't find

Before writing this up I went looking for prior art. The closest patterns in the Agent Skills ecosystem:

Anthropic's marketplace.json plus /install: version tracked in a marketplace manifest, updates happen via an external CLI command or an update_marketplace.py script. The user has to step out of whatever they were doing to learn they're behind.
skills-updater (community skill): scans your installed skills, checks each against its remote repo, and offers to update them. The closest prior art I found. But it's a separate skill you remember to run, not a check embedded in the skill you're actually trying to use.
Smithery, OpenCode, and other skill hosts: track versions for distribution but don't ship a self-check.

None of them put the check — and the install — inside the skill itself, at invocation time, against a repo-local manifest. Which is fair: it's not a big idea. It's just a pattern that package managers have had for decades, applied to a new distribution format that doesn't have one yet.

If you maintain skills and your users install them as loose files, steal this. It works for any agent whose skill format is "a folder with a markdown file and some frontmatter."

Have you seen a better workflow for keeping agent skills in sync with their source? I'd genuinely like to know. This is the best I could come up with, but that's a low bar when the format is new enough that conventions haven't settled.

Standards don't prove themselves

joost@joost.blog (Joost de Valk) — Mon, 13 Apr 2026 00:00:00 GMT

Measuring whether a standard works by checking if anyone uses it before it exists is backward. And yet that's exactly what The SEO Framework just did.

They published data showing that across six months, 57 AI bots, and 180,000 AI-related requests to their site, not a single bot requested llms.txt. Their conclusion: implementing it would be "a waste of resources." Yoast SEO and Rank Math, they imply, are selling a feature that doesn't do what it promises.

The data is solid. I have no reason to doubt the methodology. The conclusion is where it breaks.

How standards actually get adopted

Web standards don't get adopted because bots start requesting something that doesn't exist yet. They get adopted because publishers start serving something, which gives crawlers a reason to look for it, which gives more publishers a reason to serve it. That's the cycle. Someone has to go first.

XML sitemaps followed exactly this path. Google launched the protocol in June 2005. For over a year, Google was the only search engine using it. Yahoo and Microsoft didn't join until November 2006. Ask.com waited until April 2007. If you'd analyzed server logs in early 2006 and concluded "only one search engine uses sitemaps, not worth implementing," you'd have been right about the data and completely wrong about the trajectory.

IndexNow is an even more recent example. Bing and Yandex launched it in October 2021. By August 2022, over 16 million websites were publishing 1.2 billion URLs per day through it. By September 2023, that grew to 60 million websites and 1.4 billion URLs daily. Google still hasn't adopted it. If you'd waited for Google to adopt IndexNow before implementing it, you'd still be waiting. Meanwhile, Bing serves a significant and growing share of AI-powered search through Copilot and ChatGPT.

The pattern is always the same: someone builds it, someone else adopts it, adoption creates incentive, incentive creates more adoption. At no point does a standard prove itself to an empty room.

The multiplier effect

Here's what makes this particularly relevant for SEO plugins. Yoast SEO runs on over 10 million WordPress sites. Rank Math runs on over 3 million. When those plugins ship a feature, it doesn't create one data point. It creates millions of endpoints overnight.

That's the multiplier effect. SEO plugins don't just implement standards. They create the supply that gives crawlers a reason to consume them. When Yoast shipped XML sitemaps, it didn't help a few individual sites get indexed. It made sitemaps ubiquitous enough that every search engine had to support them. When Yoast added IndexNow, it turned a niche protocol into infrastructure overnight.

The SEO Framework has around 200,000 active installs. That's a solid user base, and the plugin has earned its reputation for being lightweight and well-built. But 200,000 sites choosing not to serve llms.txt has negligible impact on whether AI bots will ever look for it. The adoption question will be decided by the plugins with 10 million installs, not 200,000.

Which makes the framing of their tweet odd. They're not saying "we're choosing not to implement this yet." They're implying Yoast and Rank Math are misleading their users by shipping a feature no AI bot actually uses. And honestly, there's something to that: both Yoast and Rank Math market llms.txt with stronger claims about its current usefulness than the evidence supports.

Shipping the feature is the right call. Overselling it isn't. But The SEO Framework's response overcorrects in the other direction, dismissing the concept entirely based on the absence of adoption that hasn't had time to materialize. And it's a strong implication to make from a position that has essentially no influence on the outcome they're measuring.

The cost of waiting vs. the cost of trying

Implementing llms.txt in a WordPress plugin is trivial. You're generating a markdown file from data the plugin already has: titles, URLs, descriptions. The cost is measured in hours, not weeks. The ongoing maintenance cost is near zero. Arguably, the time spent analyzing six months of server logs to prove llms.txt is a "waste of resources" could have been spent implementing it.

The potential upside? If AI systems start consuming llms.txt (and they might, precisely because millions of sites now serve one), your users benefit immediately. If they don't, you've lost almost nothing.

Compare that to the cost of waiting. If you wait until llms.txt is proven, your users are late. They don't get the early-mover advantage. And you've contributed nothing to the ecosystem that might have made it succeed.

This isn't unique to llms.txt. The same logic applies to every emerging standard in this space.

It's not just llms.txt

The broader question isn't whether llms.txt specifically will win. It's whether we're building toward machine-readable architecture at all. And there are multiple complementary approaches, each solving a different piece of the puzzle.

Content negotiation

Cloudflare launched Markdown for Agents in February 2026, serving markdown instead of HTML when an agent sends an Accept: text/markdown header. I built the same thing independently as a WordPress plugin, with richer metadata and dedicated .md URLs. Claude Code and other AI coding tools already send this header. The infrastructure is ready. The 80% token reduction over HTML is real and measurable.

Schema endpoints and schema maps

Instead of requiring agents to crawl every page to understand your site, you can serve your entire structured data graph through a single endpoint. Yoast launched Schema Aggregation in March 2026, in collaboration with Microsoft's NLWeb project, bringing this to millions of WordPress sites. I'm proud that the company I co-founded is doing this, though I wish they'd moved faster. I've built the same concept into my Astro SEO library and have it running on this site. The approach is sound: give agents a complete, deduplicated map of your content in one request instead of making them reverse-engineer it from thousands of HTML pages.

NLWeb

Microsoft's open protocol for connecting websites with AI systems. It builds on schema.org, which LLMs already understand well. Still early, but the pieces are falling into place. A <link rel="nlweb"> tag in your HTML head, a conversational endpoint, and your existing structured data. One line of HTML and you're discoverable.

llms.txt itself

A curated, markdown-formatted index of your most important content, placed at a well-known URL. Simple, low-cost, and easy to generate automatically.

These aren't competing standards. They're layers. llms.txt helps agents find your content. Content negotiation helps them consume it efficiently. Schema endpoints help them understand its structure and relationships. NLWeb lets them query it conversationally. You don't pick one. You build toward all of them.

I've been writing about this for a while

Back in 2022, optimizing crawling for the environment showed how WordPress creates seven or more URLs for every single post, all of which get crawled, none of which add information. The follow-up proposed inverting the crawling model entirely: what if search engines only crawled URLs that sites explicitly listed in their sitemaps?

By 2025, AI systems were struggling to understand the modern web because so much of it is buried behind JavaScript rendering and div soup. The irony: llmstxt.org was already pushing for exactly the kind of machine-readable content that post argued for.

This year, the pieces are getting built. Markdown Alternate for serving markdown to agents. seo-graph for linked JSON-LD knowledge graphs. Schema endpoints and schema maps for agent discovery. NLWeb integration. Each piece is small. None of them have won yet. But they're all attempts to answer the same question: how do we make the web readable by machines, not just browsers?

The thread connecting all of this is simple: everyone who understands crawling can see that there's an opportunity to optimize how search engines and LLMs retrieve data from websites. That optimization is in the interest of the crawlers, those being crawled, and everyone in between (hosts, CDNs, the environment). The question has never been whether to build machine-readable architecture. It's how, and how fast.

Picking sides after the debate is settled

What bothers me about The SEO Framework's position isn't the decision not to implement llms.txt. Plenty of reasonable people could look at the current state and decide to wait. That's a legitimate technical judgment.

What bothers me is the framing. They're not saying "we're watching this space." They're implying the plugins that did implement it are misleading their users, positioning "we only pick sides when the data is clear" as the principled stance.

But there's nothing principled about only joining a debate after it's been settled. That's just showing up to the victory party. Standards need early adopters who are willing to invest before the returns are obvious. That's what moves the ecosystem forward. Measuring traffic to a file that barely exists yet and concluding the concept is flawed is like checking how many people show up to a restaurant before you've opened it. And concluding nobody wants to eat there.

I've been arguing that the SEO community should be playing a role in building better standards for AI systems, not waiting for the platforms to figure it out. We should be giving them clear feedback on how this could all work better. SEOs have always played a role in bettering (and worsening) the web. We should keep playing that role, preferably the first one.

The SEO Framework makes a good plugin. Their data analysis was thorough. But their conclusion says more about their approach to the industry than it does about llms.txt.

Standards don't prove themselves. People prove them by building.

Astro SEO: the definitive guide

joost@joost.blog (Joost de Valk) — Sun, 12 Apr 2026 00:00:00 GMT

In 2008, I wrote WordPress SEO: the definitive guide. It became one of the most-linked SEO articles on the internet and laid the groundwork for what eventually became Yoast SEO. The tools have changed, the web has changed, and my thinking on several fundamentals has evolved. This is the Astro version.

When I moved this blog to Astro, people asked: what about SEO? The answer is that everything I did on WordPress, I do on Astro. And because I control the entire HTML output, most of it is easier to get right. No theme conflicts, no plugin fights over head tags, no render-blocking resources injected by something I forgot I installed.

No server to compromise, no database to inject into, no login to brute force. From an SEO perspective, static HTML on a CDN is a better starting point than most CMSes will ever give you.

<div class="not-prose rounded-lg border-2 border-primary/20 bg-tertiary px-6 py-5 dark:border-accent/20 dark:bg-stone-900"> <p class="mb-2 text-lg font-semibold text-primary dark:text-accent">Want your AI agent to do this for you?</p> <p class="text-base leading-relaxed text-stone-700 dark:text-stone-300">Install my <a href="https://github.com/jdevalk/skills?tab=readme-ov-file#-astro-seo" class="text-primary underline decoration-1 underline-offset-2 hover:no-underline dark:text-accent">Astro SEO skill</a>, or point your AI coding agent at this article. Everything below is written so an agent can read it and implement it directly.</p> </div>

Here's the full stack.

1. Technical foundation

One component for all head metadata

The starting point is @jdevalk/astro-seo-graph, a package I built for exactly this purpose. The <Seo> component handles everything you'd normally scatter across your <head>: title, description, canonical, Open Graph, Twitter cards, hreflang, and the JSON-LD graph.

---
import Seo from '@jdevalk/astro-seo-graph/Seo.astro';
---

<Seo
    title="My Post | My Site"
    description="A concise description for search engines."
    canonical="https://example.com/my-post/"
    ogType="article"
    ogImage="https://example.com/og/my-post.jpg"
    ogImageAlt="My Post"
    ogImageWidth={1200}
    ogImageHeight={675}
    siteName="My Site"
    twitter={{ card: 'summary_large_image', site: '@handle' }}
    article={{ publishedTime: publishDate, tags: ['Astro', 'SEO'] }}
    graph={graph}
    extraLinks={[
        { rel: 'icon', type: 'image/svg+xml', href: '/favicon.svg' },
        { rel: 'sitemap', href: '/sitemap-index.xml' },
        { rel: 'alternate', type: 'application/rss+xml', href: '/feed.xml', title: 'RSS' },
    ]}
/>

That's the entire BaseHead.astro here (minus font preloads and analytics). It used to be 130 lines.

The component handles several SEO details automatically:

Canonical URLs are derived from Astro's site config with query parameters stripped by default (UTM tags and other tracking params don't create duplicate canonicals)
Robots meta always includes max-snippet:-1, max-image-preview:large, max-video-preview:-1 for maximum snippet sizes
Canonical is omitted when noindex is true, per Google's recommendation
Twitter tags that duplicate their Open Graph equivalents are suppressed (Twitter falls back to OG automatically)
Hreflang alternates for multilingual sites. This blog is English-only so it doesn't need them, but limonaia.house uses the same <Seo> component with alternates to tell search engines which page is the Italian version and which is the English version. Without hreflang, Google may show the wrong language version in search results, or treat the translations as duplicate content.

Auto-generated OG images

Every page gets a 1200×675 Open Graph image generated at build time. That size matters: Google Discover requires images at least 1200px wide, and the 16:9 ratio works well across social platforms. The route at /og/[...slug].jpg uses satori to render a template to SVG, then sharp to convert it to JPEG. If the post has a featured image, it's composited into the design. If not, the template fills with the title and site branding.

Why JPEG and not WebP or AVIF? Because social platforms don't reliably support modern formats yet.

The <Seo> component derives the OG image URL from the page slug automatically:

const slug = Astro.url.pathname.replace(/^\/|\/$/g, '');
const ogImage = new URL(`/og/${slug || 'index'}.jpg`, SITE_URL).toString();

No manual image creation, no missing OG images, no forgetting to update them when you change the title.

Build-time validation

The seoGraph() integration that handles IndexNow also validates your built HTML on every build. All six checks are on by default:

H1 validation: Warns about pages with zero or more than one <h1> element, a common SEO and accessibility issue that's easy to miss in templates.
Duplicate title/description detection: Checks across all built pages for duplicate <title> or meta description values. Here, it caught paginated blog pages all sharing the same title, a corpus-level SEO problem that per-page checks can't find.
Schema validation: Validates the JSON-LD structured data on every page, which brings us to the next layer.
Image alt validation: Warns about <img> tags without an alt attribute. Ran this once on my own site and got a list of 24 images across 14 posts with missing alt text that I'd missed over the years. Fixed in an afternoon.
Metadata length validation: Flags titles or meta descriptions outside SERP-safe bounds. Defaults are title 30–65 and description 70–200. Configurable per site — a personal blog with listing pages like /ask/ can loosen the title min, which is what I did here.
Internal link validation: Scans every page's <a href> values and checks them against the set of paths the build actually produced. Catches the common "I linked to /about-me but the page is /about-me/" bug, where the site "works" via a 301 redirect but wastes a round-trip on every visit. Catches typo-broken links too. Supports a skip callback for paths handled at the CDN layer (like generated sitemaps).

What the link validator found on this site the first time I ran it: a broken /about-me (no trailing slash) in one post, every pagination link in my blog archive missing its trailing slash, and five internal links to /plugins/* that were being served via a redirect because the pages had moved to a different domain. The first two took five minutes to fix, the third was a real content issue I replaced with direct links.

Beyond build-time checks, add a broken link checker to your CI pipeline. A lychee GitHub Action that runs on every push to your content files catches dead links to external URLs — which validateInternalLinks doesn't cover. A weekly scheduled run catches link rot as external sites move or disappear. Broken outbound links are a bad user experience and a negative trust signal.

2. Structured data

Most sites that have structured data at all output a flat snippet: a single Article or WebPage object. That's better than nothing, but it doesn't tell search engines or AI agents how things connect. Who wrote this article? What site is it on? What organization does the author work for?

I wrote about this in detail when I shipped @jdevalk/seo-graph-core, the graph engine that powers this site's structured data. The short version: every page outputs a linked @graph with WebSite, Blog, Person, WebPage, BlogPosting, BreadcrumbList, and ImageObject entities. They're wired together with @id references so an agent can walk the relationships.

Trust signals in your graph matter more than they used to:

publishingPrinciples tells agents where your editorial policy lives
copyrightHolder and copyrightYear communicate rights
knowsAbout helps topical authority
SearchAction on the WebSite tells agents how to search your content

You can visualize this blog's graph to see how the entities connect.

The <Seo> component takes the assembled graph as a prop and renders it as <script type="application/ld+json"> alongside all the meta tags. Everything in one place.

The seo-graph repo includes a 3,000-line AGENTS.md with recipes for fourteen site types (blog, e-commerce, vacation rental, podcast, documentation, and more), so your AI coding agent knows which entities to pick for your specific case.

3. Content optimization

This is where my thinking has changed the most since 2008.

Topics over keyphrases

The original WordPress SEO guide spent a lot of time on keyphrase optimization: pick a focus keyword, use it in your title, heading, first paragraph, meta description. That advice was correct for a world where search engines matched strings. It's much less relevant in 2026.

Modern search is vectorized. Engines and AI models convert your content into mathematical representations that capture meaning, not exact word matches. Writing about "building websites with Astro" has a much better chance of surfacing for related queries like "static site generators" than it used to. You don't need those exact phrases in your text. Exact keyword placement still has some effect, but it's far less important than covering the topic thoroughly and clearly.

That doesn't mean titles and descriptions don't matter. They do, because humans read them in search results and social shares. But optimizing them for exact keyword placement is solving a problem from 2015.

Write for humans and extraction

Readability has always mattered, but the reason has expanded. It's no longer just about keeping humans engaged. AI systems pull answers from your content, and the unit of extraction is the paragraph. A self-contained paragraph that makes its point without requiring context from surrounding paragraphs is the one that gets surfaced in AI-generated answers. If your paragraph can't stand on its own, an AI agent can't use it.

Several specific things matter more than they used to:

Lead with the point. Every paragraph should open with its most important sentence. That's what AI systems quote, and it's what L2 English readers use to decide whether to keep reading.
Keep sentences short. Sentences over 20 words are harder to parse, especially for readers who aren't native English speakers. Most of your audience reads English as a second language. Write for them.
One idea per paragraph. When a paragraph does two things, neither is extractable. Break it.
Use transitions. Words like "because", "however", "for example" tell readers (and machines) how paragraphs connect. Without them, your post reads like a list of unrelated statements.
Avoid filler. Words like "basically", "simply", "really", "just" add length without meaning. Cut them.

The same writing discipline that makes content readable to humans makes it useful to machines. If you write with an AI agent, the readability-check skill can audit your drafts against these criteria automatically.

Enforce structure at the content level

Astro's content collections let you enforce SEO fields at the schema level using Zod validation. @jdevalk/astro-seo-graph provides seoSchema which enforces title length (5-120 characters) and description length (15-160 characters):

import { seoSchema } from '@jdevalk/astro-seo-graph';

const blog = defineCollection({
    schema: ({ image }) => z.object({
        title: z.string(),
        publishDate: z.coerce.date(),
        seo: seoSchema(image).optional(),
    }),
});

If someone adds a 200-character SEO title, the build fails. That's more reliable than a runtime warning in a CMS sidebar.

The schema endpoints also include articleBody with the full text of each post (markdown-stripped, up to 10K characters), giving AI agents access to your content through structured data, not just by scraping rendered HTML.

4. Site structure

Content collections as typed architecture

On WordPress, your site structure lives in the database: posts, pages, categories, tags, custom post types. On Astro, it lives in the file system as content collections with typed schemas. Each collection is a directory of Markdown files with frontmatter validated against a Zod schema at build time.

This is stricter than WordPress taxonomies. A blog post that's missing a required publishDate or has a malformed category array won't build. The site structure is enforced by the type system, not by conventions that plugins may or may not follow.

One taxonomy, not two

Most sites should not use both categories and tags. Pick one taxonomy and commit to it. If you have both, one of them is almost certainly adding clutter without adding navigational value. I researched this and wrote a plugin about it for WordPress. On Astro, I simply didn't build tags. This site uses categories only.

Breadcrumbs linked to the graph

Breadcrumbs aren't just a visual navigation aid. In the JSON-LD graph, each breadcrumb item can reference a schema entity via @id. Here, the "Blog" crumb in every blog post's breadcrumb trail links directly to the Blog entity in the graph, not just to the /blog/ URL. That tells agents the structural relationship between the breadcrumb and the blog as a publication.

Internal linking

Internal linking doesn't need to be a manual process. Tools like Graphify can analyze your content as a knowledge graph and surface linking opportunities you'd never find by scanning posts manually. The structure of your internal links is one of the strongest signals you control. Automate the discovery; be intentional about the execution.

5. Performance

Performance is where Astro's architecture does most of the work for you.

Static by default. Every page is pre-rendered to HTML at build time. No server-side rendering, no database queries, no PHP execution. The baseline is already fast because there's nothing slow to do.

Zero JavaScript by default. Astro ships no client-side JS unless you explicitly add interactive components. Compare that to WordPress, where the average page loads dozens of scripts from themes, plugins, and third-party services.

Image optimization. Astro's built-in <Image> component generates responsive srcset attributes, converts to WebP, and adds loading="lazy" and decoding="async" automatically.

Font preloading. Preload your primary web font in woff2 format. Mona Sans is preloaded in the <head> so it's available before the first paint.

View Transitions. Astro's <ClientRouter /> with defaultStrategy: 'viewport' prefetches links as they scroll into view, making navigation feel instant while keeping the initial load minimal.

CDN headers. Hashed assets under /_astro/ get Cache-Control: public, max-age=31536000, immutable. They never need revalidation because the filename changes when the content changes. How you set these headers depends on your platform: Cloudflare Pages and Netlify use a _headers file, Vercel uses vercel.json, and other hosts typically use server config.

No-Vary-Search. UTM parameters break browser caching: ?utm_source=linkedin and ?utm_source=email are treated as different resources. The No-Vary-Search response header tells the browser to ignore specified query parameters when matching cached responses:

No-Vary-Search: key-order, params=("utm_source" "utm_medium" "utm_campaign" "utm_content" "utm_term")

Same page, different UTM tags, one cached response. Supported in Chrome, degrades gracefully elsewhere.

6. Sitemaps and indexing

Per-collection sitemaps

Astro's @astrojs/sitemap integration generates sitemaps, but the defaults are a single file with no lastmod dates. Both are easy to fix.

The chunks option splits your sitemap by content type:

sitemap({
    entryLimit: 1000,
    chunks: {
        'posts': (item) => {
            if (isABlogPost(new URL(item.url).pathname)) return item;
        },
        'videos': (item) => {
            if (/\/videos\/[^/]+/.test(new URL(item.url).pathname)) return item;
        },
    },
})

For this blog, that produces sitemap-posts-0.xml, sitemap-videos-0.xml, and sitemap-pages-0.xml (the default bucket for unmatched URLs). Per-collection sitemaps make it much easier to debug indexing issues in Google Search Console and Bing Webmaster Tools, since each collection shows separately.

Git-based lastmod

For lastmod, the serialize callback adds dates. Frontmatter publishDate only gives you a date, not a timestamp. File system timestamps reset on CI. The reliable source is git:

function gitLastmod(filePath) {
    const log = execSync(
        `git log -1 --format="%cI" -- "${filePath}"`,
        { encoding: 'utf-8' }
    ).trim();
    return log ? new Date(log) : null;
}

This gives you the actual timestamp of the last commit that touched each content file, falling back to frontmatter dates when git history isn't reliable (for example, after a bulk content migration).

IndexNow

Sitemaps are passive: you publish them and wait for crawlers to find the changes. IndexNow is the active counterpart. When you publish or update content, you push a notification to Bing, Yandex, and other supporting search engines, telling them the URL has changed.

@jdevalk/astro-seo-graph ships an Astro integration that submits all built URLs to IndexNow automatically after each build:

// astro.config.mjs
import seoGraph from '@jdevalk/astro-seo-graph/integration';

export default defineConfig({
    integrations: [
        seoGraph({
            indexNow: {
                key: 'your-indexnow-key',
                host: 'example.com',
                siteUrl: 'https://example.com',
            },
        }),
    ],
});

You also need a key verification route so search engines can confirm host ownership:

// src/pages/[your-key].txt.ts
import { createIndexNowKeyRoute } from '@jdevalk/astro-seo-graph';

export const GET = createIndexNowKeyRoute({ key: 'your-indexnow-key' });

When I first enabled this, the build pinged Bing with all 153 URLs on the site. Without IndexNow, you publish and wait for search engines to discover your changes on their next crawl, which can take days or weeks. With it, the search engine knows about your new or updated pages within seconds of the build finishing.

RSS feed

An RSS feed is still one of the most reliable discovery mechanisms. Feed readers, podcast apps, and an increasing number of AI agents consume RSS directly. Astro's @astrojs/rss package makes this straightforward: you create a route that pulls your content collection, renders each post's body to HTML, and returns a valid RSS 2.0 feed. The <Seo> component advertises it with a <link rel="alternate" type="application/rss+xml"> tag in the head so browsers and agents can discover it automatically.

Include the full post content in your feed, not just excerpts. Truncated feeds frustrate readers and give AI systems less to work with. If someone is subscribed to your feed, they've already opted in to reading your writing.

robots.txt

A dynamic robots.txt route references the sitemap index and the schema map (see next section):

User-agent: *
Allow: /

Sitemap: https://joost.blog/sitemap-index.xml
Schemamap: https://joost.blog/schemamap.xml

7. Agent discovery

Sitemaps and IndexNow help search engines find your content. Agent discovery helps AI systems understand it.

Schema endpoints and schema map

Schema endpoints serve a corpus-wide JSON-LD graph that lets an agent understand your entire site in one request. This is part of Microsoft's NLWeb specification. It's still early, but the setup is simple enough that it's worth having ready:

Each endpoint collects every entry in a content collection, builds the full JSON-LD graph for each entry, and serves the combined result as application/ld+json. This site has three: /schema/post.json, /schema/page.json, and /schema/video.json. A schema map at /schemamap.xml lists them all, like a sitemap but for structured data. The Schemamap: directive in robots.txt points agents to it. See the astro-seo-graph documentation for the full implementation.

Markdown alternates

AI agents parse markdown more reliably than HTML. Astro sites are unusually well-placed to serve it: content collections already are markdown, so there's no lossy HTML-to-markdown conversion — you hand the agent the source. astro-seo-graph ships a createMarkdownEndpoint factory that exposes a .md URL for every page (frontmatter + body + a X-Markdown-Tokens header so agents can size requests). The <Seo> component auto-emits <link rel="alternate" type="text/markdown" href="…"> so agents discover it the same way browsers discover an RSS feed.

For content negotiation — agents that send Accept: text/markdown on the canonical URL — add a Cloudflare Transform Rule that rewrites the path when the header is present. The README has the exact rule; it works on the free plan and needs no SSR.

llms.txt

The llms.txt standard provides a machine-readable summary of your site at /llms.txt. Think of it as a robots.txt for AI: it tells language models what your site is about, what content is available, and where to find it. The seoGraph() integration generates it automatically at build time from your built pages:

seoGraph({
    llmsTxt: {
        title: 'My Site',
        siteUrl: 'https://example.com',
        summary: 'What this site is about.',
    },
})

No AI bot widely requests llms.txt today. That's not the point. Standards don't prove themselves before adoption. SEO plugins that ship llms.txt across millions of sites create the supply that gives crawlers a reason to look for it.

NLWeb discovery

A <link rel="nlweb"> tag in the head points to the site's conversational endpoint, for AI agents that support Microsoft's NLWeb protocol. Early days, but the tag is one line.

8. Redirects and error handling

Redirects

When you change a URL, delete a page, or consolidate duplicate content, you need redirects. How you configure them depends on your hosting platform. On Cloudflare Pages, a _redirects file in public/ works. On Netlify, you can use the same _redirects format or netlify.toml. On Vercel, redirects go in vercel.json. The syntax differs but the principle is the same: map every old URL to its new location with a 301 status.

Keep this maintained. Every URL that ever existed and moved should have a redirect. Search engines transfer link equity through 301 redirects, and users who bookmarked the old URL still arrive where they should.

FuzzyRedirect on 404

For the URLs that slip through, the FuzzyRedirect component from @jdevalk/astro-seo-graph acts as a safety net. It fetches your sitemap, computes Levenshtein distance against the current URL, and either auto-redirects (above 85% similarity) or shows a "Did you mean..." suggestion. Catches typos and old URL patterns without maintaining a redirect table.

9. Analytics and measurement

Plausible for traffic analytics. Privacy-friendly, no cookie banner needed, lightweight script that doesn't slow down the page.

Google Search Console for indexing status, structured data validation, and search performance. The per-collection sitemaps make debugging easier: if something isn't being indexed, you can see immediately whether it's a posts problem, a videos problem, or something else.

Bing Webmaster Tools for the same reasons, and because Bing's index feeds AI products like Copilot and ChatGPT. If your content isn't indexed by Bing, it's invisible to a growing share of how people find information.

Structured data validation. Use Google's Rich Results Test and ClassySchema to verify your JSON-LD graph. Check that every @id reference resolves, that the entity relationship tree is complete, and that trust signals like publishingPrinciples and copyrightHolder are present.

The full stack

Here's what it looks like assembled:

@jdevalk/astro-seo-graph for the <Seo> component, schema endpoints, schema map, IndexNow, llms.txt, FuzzyRedirect, and build-time validation (includes @jdevalk/seo-graph-core for the JSON-LD graph engine)
@astrojs/sitemap with chunks for per-collection sitemaps and serialize for git-based lastmod
satori + sharp for auto-generated OG images
astro-pagefind for client-side search (which the SearchAction in the schema points to)
readability-check skill for AI-assisted content auditing

All of it is open source. You can see it running live on this site: view source on any page for the JSON-LD graph, check the sitemap index, the schema map, or visualize the full graph for the homepage.

Eighteen years after that first WordPress SEO guide, the principles haven't changed: make your content accessible to both humans and machines, structure it so search engines understand it, and make the whole thing fast. What has changed is that the machines reading your content are no longer just crawlers following links. They're AI agents that understand meaning, extract paragraphs, and walk knowledge graphs. The SEO stack should reflect that. This one does.

If you're building an Astro site with an AI coding agent, point it at this post and let it set everything up for you.

Shipping pieces of the machine-readable web

joost@joost.blog (Joost de Valk) — Fri, 10 Apr 2026 00:00:00 GMT

Recently I wrote that the open web doesn't need more defenders, it needs builders. That the next open web needs machine-readable architecture: content structured for machines, not just rendered for browsers. Linked knowledge graphs that search engines and AI agents can consume without reverse-engineering HTML.

That's the argument. Here's what I actually built.

The problem that started it

I was building an SEO plugin for EmDash, and the core problem was generating a valid schema.org @graph for every page. Not just a flat snippet, but a proper linked graph: WebSite, WebPage, Article, Person, BreadcrumbList, all wired together with @id references so an agent or search engine can walk the relationships.

I'd already written that logic for this blog. It worked, but it was tangled into the Astro components. When I needed the same graph logic in EmDash, the choice was: copy it and maintain two versions, or extract it into something shareable.

Before AI, "extract this into a shared library" was a decision you'd defer for months. Set up the monorepo, extract the code, write the tests, publish to npm, migrate the consumers, verify nothing broke. That's a week of work, and it's boring work, the kind that loses every prioritization fight against shipping features. With Claude Code, I went from "I should extract this" to two npm packages shipping with over a hundred tests, four migrated consumers, and CI with provenance in one session. The decision to abstract stopped being "someday" and became "right now, while I'm thinking about it."

That's the real shift AI enables. Not that it writes code faster. That it makes the right architectural decision the easy one, instead of the expensive one you keep putting off.

What seo-graph is

The result is a monorepo with two packages on npm, both MIT-licensed:

@jdevalk/seo-graph-core is the engine. Pure TypeScript, no framework dependencies. Specialized builders for the most common entities (buildWebSite, buildWebPage, buildArticle, buildBreadcrumbList, buildImageObject, buildVideoObject) and a generic buildPiece<T> that handles everything else: Person, Organization, Product, Blog, Recipe, Event, any schema.org type, with full autocomplete via Google's schema-dts.

An IdFactory keeps @id references stable across the graph. assembleGraph wraps everything into a proper JSON-LD envelope with deduplication and optional dangling-reference validation.

In plain language: you tell it what's on your page (a blog post, a person, an organization, a product) and it produces a block of structured data that search engines and AI agents can read. Every entity gets a stable identifier so agents can follow the links between them: this article was written by this person, published on this website, filed under this category. That's the knowledge graph.

@jdevalk/astro-seo-graph is the Astro integration. It ships a <Seo> component that handles <title>, meta description, canonical URLs, Open Graph, Twitter cards, hreflang alternates, and the JSON-LD graph in a single component. It also provides route factories for schema endpoints (createSchemaEndpoint) and a schema map (createSchemaMap) for agent discovery. Plus Zod helpers for content collection schemas.

The third consumer is @jdevalk/emdash-plugin-seo, the EmDash SEO plugin that started this whole thing. It imports assembleGraph from the core directly, since EmDash contributes metadata through hooks rather than templates.

Four consumers, one graph

The real test of an abstraction is whether it works for cases you didn't design for. Four consumers now use seo-graph in production:

This blog (joost.blog) is an Astro site. It uses the full stack: <Seo> in the <head>, schema endpoints serving the corpus-wide JSON-LD graph, a schema map for discovery. The BaseHead.astro component that used to be 130 lines of hand-written meta tags and JSON-LD is now a clean <Seo> call.

limonaia.house is a vacation rental site, also Astro. Completely different content type: VacationRental instead of Article. It was the first external consumer, and it plugged in with zero changes to the core or the integration. That's when I knew the abstraction boundary was right.

cocktail.glass is a cocktail recipe site. Yet another content type: Recipe with ingredients, instructions, and nutrition data. Same graph engine, same <Seo> component, different schema.org types.

The EmDash SEO plugin runs inside a CMS, not a static site generator. It doesn't use <Seo> at all. It uses assembleGraph and the GraphEntity type from the core, with its own EmDash-specific piece builders. Same graph engine, completely different integration surface.

What the output looks like

Here's what seo-graph produces for a blog post on this site. You can visualize the full graph for this post to see how the entities connect:

{
  "@context": "https://schema.org",
  "@graph": [
    { "@type": "WebSite", "@id": ".../#WebSite", "publisher": { "@id": ".../#Person" }, "hasPart": [{ "@id": ".../#Blog" }] },
    { "@type": "Blog", "@id": ".../#Blog", "isPartOf": { "@id": ".../#WebSite" } },
    { "@type": "Person", "@id": ".../#Person", "name": "Joost de Valk", "knowsAbout": ["SEO", "..."], "publishingPrinciples": "..." },
    { "@type": "WebPage", "@id": ".../my-post/", "isPartOf": { "@id": ".../#WebSite" }, "copyrightHolder": { "@id": ".../#Person" } },
    { "@type": "BlogPosting", "isPartOf": [{ "@id": ".../my-post/" }, { "@id": ".../#Blog" }], "author": { "@id": ".../#Person" } },
    { "@type": "BreadcrumbList", "itemListElement": [{ "name": "Home" }, { "name": "Blog", "item": { "@id": ".../#Blog" } }, { "name": "My Post" }] }
  ]
}

Every entity links to others via @id references. The WebSite contains a Blog. The BlogPosting is part of both its WebPage and the Blog. The breadcrumb trail links back to the Blog entity. An agent or search engine can walk these relationships and understand the full structure of your site.

The AGENTS.md in the repo has complete code examples for fourteen different site types, from personal blogs to e-commerce to vacation rentals.

Built for agents to use, not just humans to read

Schema.org JSON-LD is how the web communicates structured knowledge to machines. It's how Google understands your content. It's increasingly how AI agents will understand it too. But getting it right is tedious: the spec is massive, the @id linking is fiddly, and every framework reimplements it from scratch.

That's the problem seo-graph solves for the publisher. But there's a second problem: how does the developer's AI agent know which schema.org entities to use? If you ask Claude Code or Cursor to "add structured data to my Astro site," the agent needs to know that a blog post needs WebSite + WebPage + BlogPosting + BreadcrumbList + Person, that a vacation rental needs a VacationRental with a RentAction, that a product page needs Product with BuyAction and Offer. That knowledge lives in the schema.org spec, scattered across hundreds of pages.

That's why the repo's AGENTS.md is 3,000 lines long. It tells AI coding agents exactly which pieces to pick for every common site type. Fourteen site type recipes (personal blog, e-commerce, local business, vacation rental, podcast, documentation, SaaS, and more), the full API reference, patterns for trust signals like publishingPrinciples and copyright metadata, and guidance on actions (BuyAction, RentAction, QuoteAction) that tell agents what they can do with your content.

The repo also ships CLAUDE.md, .cursorrules, and .github/copilot-instructions.md as pointers, so whichever AI coding tool you use, it finds the guide automatically.

This is what "agent-ready" looks like at the tooling level. The library makes your content machine-readable. The documentation makes the library itself machine-readable. Agents helping agents.

seo-graph-core is deliberately small. Seven specialized builders, one generic typed builder, an ID factory, and a graph assembler. No opinions about your content model, your routing, or your framework. The dispatch logic lives in your code, not in the library. That's what lets the same core drive a blog, a vacation rental, a recipe site, and a CMS plugin without any of them fighting the abstraction.

If you're building on Astro, astro-seo-graph gives you the full integration. If you're building on something else, seo-graph-core gives you the graph engine and you wire it into your own templates.

One piece at a time

I don't think any single library fixes the open web. But the machine-readable web gets built the same way anything real gets built: one small, concrete piece at a time. This is one piece. It's on GitHub, it's on npm, it's MIT-licensed, and it's in production.

If you're building for the web and your pages don't have a linked JSON-LD graph yet, start there. The agents are already reading.

Defending the open web is not enough

joost@joost.blog (Joost de Valk) — Tue, 07 Apr 2026 00:00:00 GMT

Anil Dash recently published Endgame for the Open Web, and it's a piece worth reading. His argument: Big Tech is systematically dismantling the open web through AI scraping, API lockdowns, and the erosion of open source norms. His prescription: defend the institutions. Support the Internet Archive, donate to the EFF, volunteer for Wikipedia.

He's right about the threats. And his title, "endgame," is the right word. We're not in the opening moves. We're deep into a collapse that's already well underway, and the game board looks very different from what most people defending the open web still picture.

Let me start with what we're actually talking about. The open web is the part of the internet anyone can access without permission: websites you can visit without an app or an account, content you can link to, read, and build on. A personal blog is the open web. A Squarespace site is the open web. A WordPress site is the open web. A post that lives primarily inside a walled platform like Facebook or TikTok, where the platform controls who sees it and whether it's linkable, is not. The open web is defined by access, not by what software powers it.

That distinction matters, because Anil, and most people in this conversation, conflate the open web with open source. They're related, but they're not the same thing. Open source is about who controls the tools. The open web is about who can access the content. You can publish to the open web with proprietary software, and you can run open source software behind a login wall.

The two get conflated because for a long time they went hand in hand. If you wanted to really own your content and infrastructure on the open web, self-hosting open source software was the path of least resistance. Open source was how you made the open web yours. But that's a historical pairing, not a definition, and it's worth keeping the concepts separate when what's happening to one isn't what's happening to the other.

And it matters because the technical open web is fine. Protocols still work. You can still put up an Astro site. You're reading this on one. What's collapsing is the economic open web: the idea that you can publish independently, reach an audience, and build a sustainable business on open standards. That's the part in the endgame. Most people haven't noticed how far it's gone because the URLs still resolve.

How the endgame started

Anil documents real threats: AI scrapers that take everything and send nothing back. Traffic drops of 50%, 90% for some publishers. Conventions like robots.txt being ignored. Open APIs getting shut down. Open source projects drowning in AI-generated junk contributions.

But he mostly frames these as attacks. They're not just attacks. They're incentives. And publishers are following them exactly where you'd expect:

Bots scrape content and send no traffic back. Publishers lose their audiences. Then they lose their revenue. Then they paywall or quit.
Conventions like robots.txt get ignored. Publishers can't set terms anyone respects. Their only options are locking content behind authentication or accepting the loss.
Revenue collapses. Publishers move to Substack, Spotify, and YouTube, where the platforms monetize for them. They trade ownership for survival.
Open source projects drown in AI-generated junk contributions. They close to outsiders. Development slows. The tools powering the open web stagnate.

And there's a fifth chain that Anil doesn't mention: AI makes generic content free to produce. Just like generic code is now basically free, generic articles, summaries, and how-to guides cost nothing to generate. That collapses the value of commodity content to zero. Publishers who were already struggling to monetize now have to compete with infinite free alternatives to their work. There's an irony here: SEOs, the people who once helped grow the open web by making content discoverable, spent years flooding it with content optimized for search engines rather than readers. AI just automated what they were already doing, and made the economics of it collapse.

None of this is a prediction. A lot of it has already happened. 77% of news publishers now focus on subscriptions, and Press Gazette tracks the 100+ outlets with over 100,000 paying digital subscribers. Independent journalists are moving to Substack at scale, from Bari Weiss and Casey Newton to Jim Acosta, Chuck Todd, and Derek Thompson. Podcasters went exclusive on Spotify in nine-figure deals, leaving the open podcast ecosystem behind. Not because they opposed openness, but because openness stopped paying the bills. The open web isn't being killed by a single dramatic attack. It's emptying out, slowly, as millions of publishers individually and rationally decide that publishing openly is a bad deal.

Anil says we should fight back "with the same ferocity with which we're being attacked." But this isn't a battle. It's an exodus. And you don't stop an exodus by fighting harder.

The winners already left

Look at who's getting the AI training deals. The Financial Times, News Corp, Axel Springer, The Atlantic, Condé Nast, Reddit. Most of them already paywalled their way out of the open web years ago. They have distribution, they have legal teams, they have leverage. They're getting paid because they already left. The open web is fine if you're already not really on it. It's the people who stayed who are losing.

This was never really about the open web broadly. It's about the indie web: the solo experts, the niche publishers, the bloggers who knew more about one specific thing than almost anyone else in the world. The indie web is the part that's mostly already gone.

And the dividing line is distribution. If you already have it, you get a deal. If you don't, how do you build one now? Not by ranking in search that increasingly summarizes your content without sending traffic. Not by publishing openly in a market flooded with free AI-generated alternatives.

Maybe by moving to a managed platform that has its own audience. But then you have to ask whether you still own the training rights to your own work, or whether the platform already sold them. In 2024, Automattic sold WordPress.com and Tumblr user content to OpenAI and Midjourney for AI training. Users had to opt out after the fact, if they noticed. That's the implicit deal on any managed platform: the platform decides what your content is worth, and to whom.

AI companies are the canaries here. They've already figured out the open web isn't producing enough quality to train frontier models on, and they're paying expert contractors directly through Scale AI, Surge AI, and in-house expert networks to fill the gap. That's not a sign the open web is fine. It's a sign the collapse is already priced in, and the survivors are being picked.

Meanwhile, the biggest open source project on the web is busy suing itself

WordPress powers a huge chunk of the open web. If any project should be leading the response to these threats, it's WordPress. Instead, two of the ecosystem's biggest players have spent years suing each other.

While the open web's economics collapse and AI reshapes how content gets consumed, WordPress is still litigating trademark disputes. Not building trust infrastructure. Not making content machine-readable. Not competing with managed platforms on experience. The leadership that should be defending and advancing the open web is too busy fighting to do either, and too distracted to address the deep architectural problems that have been holding the platform back for years.

This isn't just a WordPress problem. It's a symptom. The open web's biggest institutions aren't focused on the existential threats. They're distracted by internal politics while the ground shifts beneath them.

Building what comes next

So: the indie web is mostly gone, the structural collapse of the economic open web is well underway, and defending the remains isn't a strategy. What is?

Not "defend harder" or "fund the Internet Archive more." Those are necessary, but they don't rebuild anything. The real question is what the next open web looks like: one that's actually competitive, economically sustainable, and legible to both humans and machines from the start.

Here's what that takes.

Trust infrastructure.<br /> The open web has no native way to prove who published something first, or whether content has been modified since publication. Provenance isn't a nice-to-have anymore. When AI can generate content indistinguishable from human writing, cryptographic proof of origin becomes foundational. Author credibility, entity consistency, first-party verification: these are the signals that platforms are already weighting more heavily.

Machine-readable architecture.<br /> Content on the open web needs to be structured for machines, not just rendered for browsers. Honestly, clean semantic HTML would already be a massive improvement over most of what's out there. You don't need exotic formats if your markup is actually readable. From there, proper content models, typed data, and formats that AI agents can consume without reverse-engineering HTML are the next step. And agents need to be able to act, not just read. Standards like WebMCP and WordPress's new Abilities API point in the right direction: make open source platforms' capabilities discoverable and composable by machines, not just by humans clicking through admin panels.

Competitive defaults.<br /> Open source projects need to ship with sane, performant defaults that compete with managed platforms out of the box. I've been talking about the devastating power of defaults for over a decade. Not "you can configure it to be fast," but fast by default. Not "you can add structured data with a plugin," but structured data as a core feature. WordPress became famous for its five-minute install, a defining default that set the bar for its era. But it hasn't grasped what the 2026 version of that looks like.

Openness has to pay for itself

Everything above is about how to build the next open web. But trust infrastructure, machine-readable content, and competitive defaults all assume something that hasn't been true for a while: that openness can sustain itself. It can't, not on its own. And this is the one thing the open web and open source have in common: neither of them works if the economics don't.

Open source is not just code. It is a system of incentives. And right now, those incentives don't align. FAIR, a federated package management system for WordPress built under the Linux Foundation, is a concrete case. The technical project delivered. The problem it solved was real. But when it came to hosting companies actually funding the infrastructure, the answer was no. Not because they disagreed. Because investment means cost, commitment, and risk. The current situation, however uncomfortable, was predictable enough. That's the pattern: companies benefiting enormously from open source infrastructure refusing to pay for its improvement, even when the alternative is quietly losing it.

There's a silver lining, though. At the recent Cloudfest hackathon, the TYPO3 community picked up FAIR, adapted it for their ecosystem, and is now preparing to release it. The technical work is going to live on, just not in the ecosystem it was built for. Another open source community understood what WordPress hosts wouldn't.

This same tension, of hosting companies extracting value without contributing proportionally, is what set off the WordPress lawsuit. Matt Mullenweg was pointing at a real structural problem. The mistake wasn't seeing the economic misalignment. It was turning the response into a personal and legal fight, ego-driven instead of system-driven. A solvable economic problem became an unwinnable battle of wills.

The next open web needs business models that make openness economically sustainable from day one. And it needs the companies that profit most from it (hosting companies, search engines, ad networks, AI companies) to fund it. Not as charity, but because their businesses quite literally depend on a functioning open ecosystem.

Money alone doesn't fix this, though. Funding has to come with proper governance: foundations, neutral stewardship, clear rules about what a project will and won't do. That's exactly what WordPress lacks and exactly why FAIR was built under the Linux Foundation in the first place. A well-funded project controlled by a single commercial entity isn't shared infrastructure. It's just a product with volunteers.

There are promising signs. x402, an open standard for internet-native payments over HTTP backed by Cloudflare, Coinbase, Stripe, AWS and others, is already live and processing real transactions. If bots can pay instead of just scrape, publishing openly stops being charity and starts being a business again. Anthropic is investing $100M in credits and $4M in donations through Project Glasswing to find and fix security vulnerabilities in open source software. That's an AI company putting real money back into the infrastructure it was built on. Two companies isn't a movement. But it's more than nothing, and more than most.

The open web doesn't need more defenders.<br />It needs builders.

Anil is right that something precious is at stake. Keep supporting the Archive and the EFF and Wikipedia. They matter. But defending the remains of the indie web is not the same thing as reviving it. The thing we actually cared about, independent publishers reaching audiences on their own terms, isn't coming back by being defended harder. It comes back, if it comes back at all, by being rebuilt on better foundations. Managed platforms can't do it: their business models depend on you staying, and none of them let you take your URL, your traffic, and your integrations with you. Only something open can.

None of this is hypothetical. WebMCP is being standardized. x402 is being implemented. New open source CMSs are shipping with structured content and scoped permissions out of the box. The pieces exist. What's missing is the collective will to assemble them, and the funding, with the right governance, to sustain the people doing the work.

Everything Big Tech built was possible because the web was open. Open content and open source trained their models. Open source powers their infrastructure. Open protocols let them scale. And now AI lets them build faster than ever, on the back of the same open ecosystem they've helped hollow out. The anger at that is justified. But anger alone doesn't build anything. A better open web does. Let's build it.

WordPress needs to refactor, not redecorate

joost@joost.blog (Joost de Valk) — Fri, 03 Apr 2026 00:00:00 GMT

When Cloudflare launched EmDash CMS on April 1st, the reactions came fast — from Matt Mullenweg himself, from Hendrik Luehrsen at Kraut.press, and from Brian Coords. Each piece approached EmDash differently, but together they crystallized something I've been thinking about for years: WordPress's deepest technical problems aren't at the surface. They're architectural. And the WordPress project keeps treating them as cosmetic.

A Cloudflare piece on application modernization offers a useful way to think about this. When organizations modernize legacy applications, they generally pick from three approaches: cosmetic changes (move to new infrastructure, change nothing underneath), targeted refactoring (rework the architecture in specific areas while preserving what works), or a full rewrite (start from scratch).

Most successful organizations mix and match, picking the right approach for each component based on risk and business value. The worst thing you can do is mistake cosmetic changes for real modernization.

WordPress, right now, is doing exactly that. And the responses to EmDash show why.

The surface vs. the structure

Matt's response was generous in places. He acknowledged the engineering quality and called the Skills implementation "brilliant." But his architectural arguments were almost entirely defensive. He suggested EmDash should adopt Gutenberg. He framed EmDash's sandboxed plugin model as impractical. He questioned Cloudflare's business motives.

What he didn't do was engage with the structural critique. Not once.

That's telling, because the structural critique is what every other respondent picked up on. Hendrik's piece went straight to the heart of it: Gutenberg maintains a structured block tree while you're editing, then serializes it to HTML comments in post_content when it saves. Every system that needs that structure downstream (APIs, headless frontends, AI agents) has to parse it back out again. WordPress VIP built an entire Block Data API just to avoid this. The parse_blocks() function exists because the storage model doesn't preserve what the editor already knows.

That's not a content model. That's a workaround for the absence of one.

The data model WordPress never fixed

The content storage issue goes deeper than Gutenberg. WordPress's fundamental data model dates back to its origins as a blogging platform. Every piece of content (posts, pages, products, courses, events) lives in wp_posts. Every piece of metadata lives in wp_postmeta as key-value pairs. This is essentially a schema-less design wearing a relational database as a costume.

In EmDash, each content type gets its own typed table. Custom fields become actual columns with proper types and indexes. This isn't exotic — it's how applications have stored data for decades. WordPress's approach made sense when the platform did one thing. It stopped making sense when plugins started turning WordPress into an application framework. And yet, twenty-plus years later, the schema remains essentially unchanged.

The recent deferral of WordPress 7.0 illustrates the problem in real time. The release was delayed because the team needs to revisit how real-time collaboration data is stored — the initial approach of stuffing it into postmeta wasn't going to hold up. They're now considering a custom table. This is exactly the pattern: a new feature runs into the limits of the existing data model, and the team has to work around it or pause to rethink.

What makes this worse is what the delay is for. Real-time collaborative editing — essentially Google Docs inside the block editor — will ship turned off by default. As Matt Cromwell argued in The Repository, this is a feature that belongs in a plugin: the existing collaboration plugin Multicollab has around 300 active installs, not 3 million. The vast majority of WordPress sites will never use it. Yet a major release is being delayed to re-architect the database layer for this, while the architectural work that every site would benefit from continues to wait.

The deferral post itself is remarkably candid about the cost. WordPress sites are "generally very read-heavy," it says, but collaboration "inherently involves a writing state." The HTTP polling approach is described as a "lowest-common-denominator" mechanism whose "broad compatibility comes at the cost of relative inefficiency." Hosts need time to "monitor requests to the sync endpoints" and "perform profiling." This is core telling hosts: we're shipping a feature that fundamentally changes how your servers behave, but don't worry, it's off by default. That's not a feature launch. That's a liability disclaimer.

And it gets worse. The core team is now publishing guides on building custom sync providers — requiring separate WebSocket servers and custom infrastructure — with WordPress VIP's implementation as the reference. This is enterprise hosting infrastructure being built into core because a handful of VIP clients need it. That's not an open source project serving its community. That's a product roadmap shaped by the needs of one company's hosting platform. Ironic, given that Matt's criticism of EmDash was that Cloudflare built it to sell their services.

But zoom back out, because the prioritization problem and the technical problem are two sides of the same coin. The reason a single database table can delay a major release is that the data model was never designed to evolve. You can put a React admin on top of wp_posts and wp_postmeta. You can add a block editor. You can ship full-site editing. The underlying data model doesn't care. It's still the same architecture, just rendered differently.

The plugin trust problem

Matt argued that plugins having full access to WordPress is a feature, not a bug, and that sandboxing "breaks down as soon as you look at what most WordPress plugins do."

This is like arguing that because some mobile apps need camera access, every app should get root access to the phone. Yes, some plugins need broad capabilities. That's an argument for a granular permission system, not for continuing to give every plugin the keys to the entire database — and the entire filesystem, since a plugin runs with the same permissions as the PHP process. Every modern platform (iOS, Android, browser extensions, cloud functions) moved to scoped permissions years ago. WordPress's full-trust model isn't a philosophical commitment to openness. It's a security debt that compounds with every plugin install.

What the insiders see

Perhaps the most telling response came from Brian Coords, an Automattic employee. His observation: WordPress has had the PHP infrastructure for custom post types and fields since version 3.0 in 2010, but has never shipped a core UI for managing them. Sixteen years later, the community still depends on third-party plugins for something that should be a core capability.

Brian framed this as a prioritization problem, not an engineering one. He's right, and that makes it worse. It means WordPress could have built this and chose not to. There's a pattern here: architectural improvements get proposed, sometimes even prototyped, but they never get prioritized. The features that move forward are the ones with a direct path to revenue or a visible demo. Modernizing the boot process, adding an autoloader, fixing the data model — none of that makes for a good keynote slide.

Brian also identified what he called "decision fatigue": WordPress now offers so many layered approaches to the same problems (core blocks, patterns, block styles, theme.json, custom CSS, classic themes, block themes) that developers spend more time choosing how to build something than actually building it. For AI agents, which can't navigate that kind of ambiguity the way experienced developers have learned to, this is a hard wall.

Refactor, don't redecorate

I didn't write about EmDash because I think WordPress should die. I spent fifteen years building Yoast SEO, and I chair Post Status, a foundation for the WordPress professional community. I care about this ecosystem. But caring about it means being honest about where it is.

WordPress doesn't need to become EmDash. A ground-up rewrite of the platform that still powers more of the web than any other CMS would be reckless. But it does need real refactoring: targeted, meaningful architectural changes that address the structural problems rather than just painting over them.

Four changes would transform WordPress without breaking it.

Fix the data model

Replace wp_posts and wp_postmeta with proper typed tables per content type, while preserving the existing APIs. This sounds radical, but it's been done before, in the WordPress ecosystem itself. When we launched Indexables in Yoast SEO in 2020, we replaced a sprawling mess of postmeta queries with a single, properly typed table. All public-facing APIs stayed the same. The data model underneath changed completely. It was hard. But it can be done, and it's easier at the core level, since core controls the schema. Automattic proved this again with WooCommerce's migration from its legacy data model to High-Performance Order Storage — a tested, backwards-compatible approach to exactly this kind of schema change.

The first step would be a real database abstraction layer: a "WP-DBAL" that lets wpdb move to a saner paradigm with a compatibility layer underneath. Right now, most plugin authors still use dbDelta to execute raw SQL when they need custom tables. That's how thin the current abstraction is. Ari Stathopoulos, whose SQLite compatibility layer made WordPress Playground possible, has already proven that WordPress can run on a completely different database engine. A proper DBAL would take that same principle further, making the schema migration straightforward without breaking backwards compatibility.

Store content as structured data

Stop serializing blocks to HTML comments in post_content. Store them as JSON instead, or at minimum alongside the HTML. The block editor already maintains a structured tree while editing; the problem is that it throws that structure away on save. Keeping it as JSON means APIs, headless frontends, and AI agents can work with WordPress content natively, without reverse-engineering structure out of markup. I opened a Trac ticket seven years ago just to allow caching of parse_blocks results — a band-aid for the performance cost of repeatedly re-parsing that serialized content. It's still open. This is the single biggest unlock for making content machine-readable, and it's what EmDash does with Portable Text.

Scope plugin permissions

Move from a full-trust plugin model to a permission-scoped execution layer. Not every plugin needs access to everything. A contact form plugin doesn't need to read the users table. A caching plugin doesn't need to write to the posts table. Every other modern platform figured this out years ago. WordPress can too.

This is the biggest lift of the four. WordPress's entire plugin ecosystem was built on the assumption of full access, and retrofitting a permission model without breaking thousands of existing plugins is a genuinely hard problem. But hard isn't the same as impossible, and deferring it indefinitely isn't a strategy. It's just accumulating risk.

Modernize the PHP foundation

WordPress still doesn't have an autoloader. Every file is loaded via manual require calls, a pattern that modern PHP left behind years ago. At Yoast, we built a fully working, backwards-compatible autoloader for WordPress core seven years ago. In 2024, Ari Stathopoulos proposed an even simpler approach in a formal core proposal. Neither made it into core. An autoloader is table stakes for any modern PHP project. It would unlock proper namespacing, reduce memory usage, and make the codebase navigable for both humans and AI agents. This is the lowest-risk, highest-leverage change on this list, and the fact that it hasn't happened yet says everything about WordPress's priorities.

None of this is theoretical. The community has shown, repeatedly, that it can be done. The pattern is always the same: working code, proven approach, no action.

These aren't small changes, but they're not a ground-up rewrite either. They're the refactoring work that WordPress has been deferring for more than a decade.

The stakes

The risk isn't that WordPress changes too much. The risk is that it keeps treating surface-level changes as if they were modernization. Gutenberg was genuine modernization when it was conceived, but after eight years it's still not finished, and the foundation underneath it never changed. The environment has changed. The way we build software has changed. WordPress's architecture needs to change with it, or the ecosystem that depends on it will look elsewhere. Many already are. I wrote recently about how the CMS market itself may not be growing anymore.

People don't want a CMS — they want a website, and the ways to get one have multiplied far beyond the traditional CMS category. The competitive pressure WordPress faces isn't just from other CMSes. It's from platforms that skip the concept entirely: Shopify, Substack, static Astro sites, AI-generated sites, and now EmDash. The market is fragmenting, and consumer expectations are shifting toward simpler, more opinionated tools.

That competitive pressure makes the architectural stagnation doubly dangerous. WordPress isn't just falling behind technically while competitors in its own category innovate. It's falling behind while the category itself is shrinking. When a company the size of Cloudflare looks at the CMS landscape and decides to build a new one from scratch rather than build on top of WordPress, that's not a side project. That's a market signal.

I called EmDash the most interesting thing to happen to content management in years. Not because of the technology stack, but because it's designed for how we actually work now: AI agents building sites, structured content that machines can parse, deployment at the edge. WordPress is postponing a major release for weeks over a single database table. In that same timeframe, one developer at Cloudflare built an entire CMS from scratch — with structured content, scoped plugins, and edge deployment out of the box. That's the gap. WordPress could have owned that future. Instead, Cloudflare saw the opening that WordPress's architectural stagnation created, and walked right through it.

But it's not too late. WordPress still has the largest install base, the biggest community, and decades of institutional knowledge. The talent is there. The ideas are there. The working code is there. What's missing is the willingness to prioritize the foundation over the facade. If WordPress starts making the right architectural decisions now, it can still catch up. But the window is closing.

EmDash: a CMS built for 2026

joost@joost.blog (Joost de Valk) — Wed, 01 Apr 2026 00:00:00 GMT

import { Image } from 'astro:assets'; import seoPanelImage from './images/emdash-seo-panel.png'; import adminImage from './images/emdash-admin.png';

The way we build software changed forever at the end of 2025. Suddenly our AI agents went from smart auto-complete to actually being able to develop software. We stopped writing boilerplate and started describing intent. But the tools we build with haven't caught up, especially content management systems. Most CMSes were designed for a world where humans click through admin panels, plugins run with full trust, and deployment means uploading files to a shared host.

That world is gone. And today Cloudflare launched something built for the one we're actually in. And no, despite it being April 1st, it's not a joke.

EmDash is the most interesting thing to happen to content management in years. Not because it's built on Astro (though it is), but because it's the first CMS designed from the ground up for how we work in 2026: AI agents building sites, structured content that machines can parse and manipulate easily, and deployment at the edge.

What EmDash actually is

EmDash is a full CMS built on Astro. End-to-end TypeScript. Deploys to Cloudflare Workers, Netlify, or Vercel. SQLite locally, Cloudflare D1 in production. Images on disk or R2/S3.

But the tech stack isn't the point. The design philosophy is.

Every architectural decision in EmDash seems to have been made with the same question: "What if an AI agent needs to do this?" Content is stored as portable text — structured JSON, not HTML strings — which means an agent can read, modify, and generate content without parsing markup. Custom content types get their own database tables with typed fields, so an agent can reason about the schema programmatically. There's an MCP server for direct CMS interaction, a CLI that outputs JSON, and documentation specifically structured for AI consumption.

Editing happens on the frontend. You see your actual site while you edit it, not a disconnected admin panel (though you can edit there too). Internationalization is built in from day one. So is redirect management, full-text search, and core SEO functionality.

And it looks a lot like WordPress, which will make people moving over feel right at home. There's even a complete guide for porting WordPress themes, with mapping tables from WordPress template tags to EmDash API calls that are practically designed to be fed to an AI agent.

The plugin security model is the real story

Any CMS lives or dies by its plugin ecosystem. The challenge has always been balancing extensibility with security: the more a plugin can do, the more damage a bad one can cause.

EmDash takes a fundamentally different approach. Each plugin runs in an isolated worker environment with granular permissions. A plugin has to explicitly request access to content, network calls, or specific APIs. The UI layer is defined through a JSON schema similar to Slack's Block Kit, which means plugins can't inject arbitrary HTML or JavaScript into the admin.

There's a plugin marketplace too, though currently API-only. Submissions go through automated security scanning powered by Workers AI, with Llama Guards handling content classification. It's not perfect — automated scanning never catches everything — but it's a meaningful layer of protection built into the platform from the start. And if you're a WordPress plugin developer, there's a guide for porting plugins that maps WordPress hooks and APIs to their EmDash equivalents.

What I think is genuinely strong

It's agent-native, not agent-compatible.<br /> Most CMSes are bolting on AI features — a chatbot here, a content generator there. EmDash is different: the entire architecture assumes AI agents are first-class users. The MCP server means Claude, Cursor, or any agent can create content types, manage entries, configure plugins, and deploy — all programmatically. The CLI outputs JSON. The documentation is structured for machine consumption. Round-trip markdown support means you can export content, edit it in any tool (or let an agent edit it), and import it back without loss. This isn't an AI feature. It's an AI-native CMS.

The AI site generation shows where this is heading.<br /> EmDash includes a playground that spins up full sites from a prompt: theme, content structure, sample data, all generated by models running on Workers AI. Today it's a demo tool. But it shows the trajectory: describe what you want, get a working site. That's not a gimmick when the entire CMS is structured data an agent can manipulate.

Authentication uses passkeys by default.<br /> No passwords to leak, no brute-force attacks to defend against. Combined with pluggable SSO and built-in role-based access control, the security posture is strong out of the box.

The SEO foundation is solid.<br /> Built-in SEO controls, metadata hooks for plugins, structured content that search engines can parse cleanly. It's not a full SEO suite, but the architecture makes it straightforward to build one.

</div> <Image src={seoPanelImage} alt="EmDash's built-in SEO panel showing fields for SEO title, meta description, canonical URL, and a noindex toggle" class="rounded-lg border border-slate-200 dark:border-slate-700 shadow-card w-full cursor-zoom-in" data-zoomable /> </div>

What's still unproven

Let's be honest about the challenges.

EmDash was built over two months using AI coding agents. That's remarkable speed, and it shows what's possible when you build on a modern foundation like Astro without decades of legacy to navigate. But a CMS is a long-term commitment, and two months of AI-assisted development is not the same as two months of battle-testing in production. The real test starts now.

The plugin ecosystem is empty. WordPress has over 60,000 plugins. EmDash has a marketplace architecture but no community yet. History shows that CMS adoption is driven more by available plugins and themes than by technical merit. Ghost, Craft, Statamic; all technically excellent, none have built the ecosystem needed for broad adoption. EmDash will need to solve that same chicken-and-egg problem.

The open-source story needs clarity. It's MIT-licensed, which is great. But it's still a Cloudflare project, and developers will want to see long-term governance commitments before investing their time.

Deployment is flexible: one click deploys to Cloudflare Workers, Netlify, or Vercel, and it's supported on any modern hosting platform. That said, the Cloudflare integration is the most polished (D1 for the database, R2 for storage), and it'll be worth testing the other hosts to see how well the multi-platform story holds up in practice.

Why I'm going to build on it

Every CMS generation reflects the tools used to build with it. WordPress was shaped by FTP, PHP, and shared hosting — and it shows. Squarespace by drag-and-drop visual builders. The next generation will be shaped by AI agents, and the CMS that wins will be the one those agents can work with most effectively.

That's what EmDash gets right. Structured content an agent can read and write natively. A typed schema it can introspect. An MCP server it can talk to directly. A plugin system with clear boundaries it can develop within safely. This isn't a CMS with AI features added on top. It's a CMS where AI is a first-class builder.

When I moved this blog to Astro, I thought I'd left CMSes behind. Then I spent an evening building an EmDash theme for this site and started migrating content — because a CMS that an AI agent can build on this naturally is hard to resist. But the bigger picture is agencies building client sites, creators who need multi-user workflows, anyone building more than a static site. For them, a CMS isn't optional. The question is what a good one looks like when your primary development partner is an AI agent.

EmDash is my answer. It's built on Astro, deploys to Cloudflare, and adds the CMS layer with the kind of architecture I'd design if starting from scratch: sandboxed plugins, structured content, TypeScript throughout, edge deployment. But what seals it is that I can point Claude at it and say "build me a theme", and the structured documentation, typed APIs, and MCP server mean it actually can.

I'm planning to develop on and with EmDash, and I'll share more about what I'm building as it takes shape. For now, go look at EmDash and form your own opinion. The launch is today, and regardless of where it ends up, it's asking the right question: what should a CMS look like when AI agents are doing most of the building?

<dialog id="img-zoom-modal" style="padding:0;border:none;background:transparent;max-width:90vw;max-height:90vh;margin:auto;outline:none;"> <img id="img-zoom-target" style="max-width:100%;max-height:90vh;border-radius:8px;display:block;" alt="" /> </dialog> <style>{#img-zoom-modal::backdrop { background: rgba(0,0,0,0.7); backdrop-filter: blur(4px); }}</style>

Ask Joost: AI-powered answers from my blog

joost@joost.blog (Joost de Valk) — Thu, 26 Mar 2026 00:00:00 GMT

I have written quite a bit. Over the years, this blog has accumulated posts about WordPress, SEO, open source governance, CMS market share, and plenty more. Finding the right post for a specific question means searching, skimming, and hoping the title matches what you're looking for. That's not great.

So I built Ask Joost: a page where you can ask a natural language question and get a direct answer, sourced from my blog posts.

Want this on your own site? I've open sourced it on GitHub.

How it works

The system has three layers: a build-time index, hybrid retrieval, and an LLM that generates grounded answers.

1. A search index built at deploy time

Every time the site builds, a script scans my content directories for blog posts, pages, and videos. It extracts titles, descriptions, categories, publication dates, and full text, then generates a searchable index that ships with the site as a static JavaScript module.

That index also includes embeddings, so there’s no separate database or vector store to maintain. And importantly, it skips draft: true content, so unpublished posts never make it into the public Ask index.

For video content, the index can also pull in generated transcripts. That makes videos searchable too, though they’re treated as weaker sources than written posts and pages.

2. Hybrid retrieval: keywords + embeddings + a bit of query rewriting

When you ask a question, it hits the /ask endpoint — a Cloudflare Pages Function. Retrieval uses a blend of keyword search and semantic search.

Keyword scoring tokenizes your query and scores documents based on term frequency, with extra weight for matches in titles, descriptions, keywords, and URLs.

Semantic search uses Cloudflare Workers AI’s bge-base-en-v1.5 embedding model. At build time, each indexed document gets embedded. At query time, the question gets embedded too, and cosine similarity finds content that’s conceptually related even when the exact words differ.

On top of that, the retrieval layer does a little bit of query normalization. It expands common aliases and shorthand, things like wp to WordPress, or gutenberg to gutenberg block editor and block editor to the same, so short or informal queries still find the right content.

For follow-up questions, it also augments vague queries with the previous turn. So if you ask “what about governance?” after a WordPress question, retrieval gets a bit more context before the model ever sees anything.

Each indexed document also has a search weight. Pages get a slight boost, videos get a slight demotion, and blog posts sit in the middle. That helps the system prefer cleaner, more authoritative written sources without making videos undiscoverable.

Individual posts can also override their default weight by setting searchWeight in their frontmatter. A value above 1.0 promotes a page in results; a value below 1.0 demotes it. This is useful when you know a particular post is the canonical answer on a topic, or when an older post has been superseded and shouldn't rank as highly anymore.

3. An LLM that generates the answer

When the Ask page calls the endpoint, it uses streaming generation by default. The top search results are packaged up as context and sent to Llama 3.3 70B running on Cloudflare Workers AI.

The prompt keeps the model on a short leash:

answer only from the provided context;
say so when the site doesn’t contain enough information;
always cite sources with markdown links;
prefer newer posts when my thinking has changed over time;
treat vague follow-up questions as part of the ongoing conversation.

Each source passed to the model includes its content type and publication date. That gives the model enough context to distinguish between “this is what Joost thought in 2021” and “this is probably his current view.”

If the model fails, times out, or returns something unusable, the endpoint falls back to a deterministic summary built from the search results. So the system degrades gracefully instead of just erroring out.

Better sources, better answers

One thing I cared about a lot was source quality. Not all content is equally good source material. Pages like “About” are often more authoritative than an old blog post. Video transcripts are useful for retrieval, but they’re rougher as quoted sources because they’re derived from captions. So the system uses them for discovery, while still preferring cleaner written sources in the final answer when possible.

The source list shown below each answer is also filtered down to the posts the model actually referenced. The function parses the markdown links in the answer and only shows those sources, instead of dumping every item that happened to be in the retrieval context.

Each visible source can also include a content-type label and publication date, which makes it easier to see whether you’re looking at a page, a blog post, or a video, and whether the source is current or old.

Streaming and conversation support

Answers stream in as they’re generated, so you don’t just sit there waiting for the full response to appear all at once. That improves perceived speed a lot, especially for longer answers.

The page also supports follow-up questions within the same conversation. It keeps a short exchange history client-side, sends it along with new requests, and uses a stable session ID so Cloudflare can keep the conversation pinned to the same model instance.

That last part matters because Workers AI supports prompt caching via session affinity. In practice, that means the system prompt and recent conversation history can stay hot in memory, making follow-ups faster and cheaper.

Why this architecture

I wanted something that:

Has no ongoing cost at rest. The search index is static. The function only runs when someone asks a question.
Doesn’t require a vector database. For a site of this size, embeddings can just live in the generated index.
Degrades gracefully. If the AI layer fails, there’s still a deterministic fallback.
Stays inspectable. The retrieval logic is straightforward enough to debug, tune, and reason about.
Is NLWeb-compatible. The /ask endpoint follows the NLWeb protocol, so agents and tools that speak NLWeb can query the site directly.

This is one of those cases where “boring architecture” is a feature. There’s no orchestration layer, no separate retrieval service, and no complex storage stack — just a static site, a generated index, a function, and Workers AI.

The tech stack

Astro for the static site
Cloudflare Pages Functions for the /ask endpoint
Cloudflare Workers AI
- @cf/baai/bge-base-en-v1.5 for embeddings
- @cf/meta/llama-3.3-70b-instruct-fp8-fast for answer generation
A build-time indexing script that generates the searchable index, embeddings, metadata, and transcript-backed content from markdown

The code is pretty small. The endpoint logic is now split into focused modules for config, retrieval, and generation, but it’s still a lightweight setup rather than a framework-heavy one.

Observability and tuning

As soon as you build something like this, you start wanting ways to inspect what it’s doing.

So the Ask endpoint also has a debug mode for inspecting retrieval results, score breakdowns, and timing. That makes it easier to tune ranking, understand why a query matched a certain document, and test improvements without guessing.

That’s been useful while tuning things like alias expansion, follow-up handling, source extraction, and weighting between pages, posts, and videos.

Use it on your own site

I’ve extracted the core into an open source package: ask-endpoint. It gives you a drop-in NLWeb-compatible /ask endpoint for markdown-based sites on Cloudflare Pages.

The setup is straightforward: point it at your content, generate the index during build, add the function, and connect a Workers AI binding. The README has the full walkthrough.

Try it

Head to /ask-joost/ and ask something. Try “do you think I need a CMS?”, “what happened with WordPress governance?”, or “how do you think AI affects SEO?” and see what comes back.

The answers still aren’t perfect. They’re constrained by what I’ve written, and the model can still miss nuance. But for a relatively small, low-cost system layered on top of a static site, it’s remarkably useful, and a lot more usable than making people dig through archives by hand.

Do you need a CMS?

joost@joost.blog (Joost de Valk) — Sat, 21 Mar 2026 00:00:00 GMT

For twenty years, "I want a website" meant "I need a CMS." WordPress, Joomla, Drupal: the conversation was always about which one. That framing is outdated. People never wanted a CMS. They want a website.

The real competitive landscape

The ways to get content on the web have multiplied. Shopify if you're selling something. Squarespace or Wix if you want something that looks good without touching code. Substack if you just want to write. A static site generator if you want performance and control.

CMS market share conversations obsess over WordPress vs. Drupal vs. Joomla. I know, because I've been tracking those numbers for years. But looking at those numbers, the real movement isn't between CMSes. It's away from CMSes entirely. The competitors that are actually growing are the ones that aren't traditional CMSes at all. I wrote about this in 2022 when WordPress' market share first started declining. The winners then were Wix and Squarespace. That trend has only accelerated.

My own answer: I don't need one

I ran joost.blog on WordPress for years. I built Yoast SEO. I'm probably the last person you'd expect to leave WordPress behind (even if only for a few sites). But when I relaunched this site recently, I moved to Astro on Cloudflare Pages.

The site is now a collection of Markdown files that get built into static HTML and deployed to a global CDN. No database. No server. No PHP. I still have everything that matters: full-text search, comments, structured data, RSS, auto-generated OG images. What I dropped was the overhead.

But what about SEO?

I built Yoast SEO, so you'd think this is where a static site falls short. It doesn't. Everything Yoast SEO does on WordPress, I can do in Astro. XML sitemaps, meta tags, canonical URLs, Open Graph tags, structured data with full JSON-LD schema graphs, auto-generated social share images: it's all there. In fact, it's easier to get right on a static site because you control the entire HTML output. There's no theme or plugin conflict messing with your head tags. No render-blocking resources injected by something you forgot you installed. What you build is what gets served.

The SEO features that a CMS plugin provides aren't magic. They're HTML output. And any modern static site generator can produce that same HTML, often cleaner.

What I gained

The result aligns with something I wrote about last year: the web works best when you keep it simple. Clean HTML that every crawler can read, including AI systems that don't execute JavaScript at all. Near-zero security surface. No updates to manage. No plugins to keep compatible. Hosting costs that round to zero. Page load times that are hard to beat with any dynamic setup.

This isn't about nostalgia for a simpler web. It's about recognizing that for a site like mine, a blog with some static pages, the complexity of a CMS was solving problems I didn't have while creating problems I did.

Where you genuinely need a CMS

Let me be clear: there are real use cases where a CMS earns its complexity. If you have a team of editors who need to collaborate on content daily, with roles, approval chains, and scheduling. If your content model is highly dynamic and relational. If you're running e-commerce, membership, or LMS functionality where the site is the application. If your content needs to change per visitor through personalization.

I'm building Rondo on WordPress for exactly this reason. WordPress works well as an application framework, and Rondo needs the kind of dynamic, user-facing functionality that a static site simply can't provide. These aren't edge cases. They represent a lot of websites. But they don't represent most websites. Most websites are a handful of pages and maybe a blog.

The editing question

The last real argument for a CMS on simpler sites is the editing experience. "My client can't edit Markdown files and commit them to a git repository." That's a fair point. Today.

But think about what's already changing. If you can tell a chatbot "update the opening hours on my contact page" or "publish a post about our spring menu" and it edits the file, commits, and deploys, what's the admin panel for? I built this entire Astro site with AI assistance. The next step, editing content through conversation, is not a big leap. It's a small one.

The CMS's visual editing interface was a solution to a human limitation: most people can't (or don't want to) write code to update their website. AI is removing that limitation. Not in some theoretical future, but now. When editing a static site becomes as easy as sending a message, the CMS's core advantage for the majority of websites disappears.

So what's a CMS actually for?

A CMS still has a real future in genuinely complex scenarios: multi-user collaboration, dynamic content, application-level functionality. But for the millions of sites that are essentially "some pages and maybe a blog"? The answer to "do I need a CMS?" is increasingly: no. You need a website. And there are more ways than ever to have one.

AI optimization is replaying early SEO, just faster

joost@joost.blog (Joost de Valk) — Wed, 11 Mar 2026 00:00:00 GMT

If you were around for early SEO, the current AI-content wave should feel familiar. The difference is speed: what used to take years now takes weeks.

The first era of SEO was not subtle. Keyword stuffing, link farms, doorway pages, directories: entire businesses existed to game ranking systems rather than serve users. I've lived that age and it wasn't pretty. Yeah, sure, for a while it worked. Then search engines evolved, penalties got harsher, and the arms race shifted toward quality signals.

We're replaying that cycle with AI content. Same incentives, same tactics, same outcome. The only real change is the feedback loop: it's shorter, and far more volatile.

The early-SEO playbook we're repeating

If you strip away the tooling, the pattern is simple: when distribution is cheap and ranking signals are gameable, people will try to scale exploitation before they scale usefulness.

Early SEO looked like this:

Scale over substance. Thousands of pages built to match query patterns, not solve problems.
Signal manipulation. Keywords, anchor text, link schemes, directory listings.
Thin content. Minimum effort to get indexed; maximum volume to get clicks.

The most important lesson from that era wasn't "SEO is bad." It was that any system measured by ranking will be gamed until quality signals improve.

The AI-era equivalents

Now we have a new way to manufacture content, and the incentives are even stronger:

Prompt-spun farms. Entire sites generated at near-zero cost, optimized for long-tail queries.
Programmatic SEO on steroids. Instead of templated pages for "best X in Y," we now generate original-ish text for each variation.
Agentic automation. Teams running content factories like software pipelines: ideate → generate → publish → iterate.
"AI SEO" as the new link building. Everyone is trying to reverse-engineer how large models cite sources and surface results.

The tactics changed. The intent didn't. It's still the same old game: grabbing a quick win from a distribution loophole before the algorithms get smart enough to close it.

Why it's faster this time

In early SEO, the production bottleneck was human labor. That kept the cycle slow. In AI, the bottleneck is no longer writing; it's verification and trust.

That changes the tempo:

Generation cost is near zero. You can spin a thousand pages before lunch, heck, before you've had your first coffee.
Iteration is instant. Change a prompt, re-run the system, ship again.
Platforms can detect faster, but spammers can adapt faster too. Every "penalty" is a new dataset to train against.
Feedback loops collapse quality. Models are trained on outputs, which means mediocrity becomes input. That creates a race to the bottom unless quality signals are explicit.

When content can be mass-produced and mass-reproduced, the only durable advantage is proven value.

What platforms will do (and already are doing)

Search engines and LLM platforms aren't blind to this. We're already seeing the direction of travel:

Stronger quality signals. Depth, originality, and usefulness are weighted higher than keyword coverage. See Google's recent Discover update, or this video from Glenn about the December update.
Provenance and identity. Author credibility, entity consistency, and first-party data matter more. Note how this Google documentation was updated in in Feb 2026, specifically targeting bylines and author reputation.
Downranking boilerplate. Templates, paraphrases, and thin content are being actively filtered.

This is the same arc as early SEO, just in fast-forward. The platform's long-term incentive is to improve the user outcome, even if it means burning a lot of low-value content along the way.

What actually works for builders & publishers

If the content arms race is speeding up, the only sustainable strategy is to produce things the algorithm wants to trust.

That means content that is earned:

Original data. Benchmarks, experiments, research, or unique datasets.
Real-world experience. Advice that only exists because you've done the work.
Insight synthesis. Connecting ideas in a way that feels new, not rephrased.

And it means building distribution you control:

Email, communities, product surfaces, and direct relationships.
A recognizable voice that makes people search for you, not just your topic.

SEO still matters, but it becomes the output of being the best source, not the goal itself. Note that I'm not saying you can't use AI for this, of course you can, and you should.

A note on Technical SEO

This, by all means, does not mean we shouldn't be playing with new technical optimizations, figuring out what works with all the new crawlers going around. We shouldn't draw conclusions on them too quickly and we should give the platforms clear feedback on how we think this could all work better. SEOs have always played a role in bettering (and worsening) the web, we should keep playing that role.

A practical playbook

If you're publishing in this environment, here's what will keep working even as platforms tighten the screws:

Create proof-of-work content. Ship real case studies, benchmarks, real results, public experiments.
Go deep, not wide. Fewer, higher-quality pieces that build topical authority.
Build distribution you own. A reader base is more defensible than a ranking.
Tie content to product value. If your writing is inseparable from what you build, it's harder to commoditize.
Embrace editorial scarcity. Don't ship more just because you can; ship better because you must.

The real lesson

The lesson from early SEO wasn't "never optimize." It was "optimize for users, because the platform will eventually force you to."

AI simply accelerates that reckoning. The penalty for low-value content arrives faster, and the reward for real value compounds sooner.

If you're building content strategy in 2026, the winning move is the same as it was in 2006, just faster:

Make something that deserves to be ranked.

FAIR, WordPress, and Knowing When to Stop

joost@joost.blog (Joost de Valk) — Thu, 26 Feb 2026 00:00:00 GMT

Over the past year, we, Karim & Joost, have written and talked extensively about the challenges facing WordPress. We proposed a path forward, laid out what should be on the roadmap, and been critical. At times, very critical. Of leadership, of governance, of the way decisions are made and enforced. We stand by that criticism.

Matt’s way of operating has, in our view, been harmful. It has led to a fractured community. It has created distrust. It has made contributors feel unsafe or unheard. That is not good for WordPress. It is not good for open source. And it is not something we should normalize.

But something else has become clear to us over the past months.

Even when you believe change is necessary, even when you build a credible alternative, you still need a broader ecosystem willing to step up.

And that’s where FAIR stops as far as WordPress goes.

Why FAIR exists

FAIR was never about “taking over” WordPress. It was never about forking, nor did it exist for the sake of ego or power. It was a response to a structural problem.

The WordPress ecosystem has grown into a massive economic engine. Hosting companies, agencies, plugin developers, SaaS providers, thousands of businesses rely on it. Those of us who believe in it think of it more as an open web operating system than a simple CMS. Yet governance and infrastructure remain tightly controlled. That imbalance creates risk.

When key infrastructure or distribution mechanisms are dependent on a single commercial entity, the entire ecosystem becomes fragile. That fragility became impossible to ignore.

FAIR, launched under the Linux Foundation, was an attempt to create a neutral, community-governed package manager and infrastructure layer. The goal was stability. Predictability. Shared stewardship. A model that could reduce systemic risk and ensure long-term independence.

In a very short period of time, the technical project went from a proposal to an effort involving a larger team and us, and then to an actual working model. Over the second half of the year, the project achieved technical success. Today, it is a viable technical alternative to the problem. But success isn’t just measured in technical success or the amount of effort.

It is a serious effort. Thought through. Properly structured. Not a Twitter thread. Not a protest. A working technical achievement and a real proposal for a model that could work well for all involved parties. But proposals need backing.

The hard reality: hosts don’t want to invest

In recent months, we’ve had many conversations with hosting companies and other large ecosystem players. What became increasingly clear is this: they do not want to invest in this kind of solution.

Not because they love the current situation. Not because they agree with everything that’s happened. But because investment means commitment. It means cost. It means stepping into political tension. And most of all, it means risk. The current situation, however uncomfortable, is predictable enough. Changing that requires money and coordinated effort. And that willingness simply isn’t there.

You cannot build alternative infrastructure of this magnitude on goodwill alone. You need adoption and funding. You need shared responsibility. You need large players willing to say: “Yes, we will carry part of this.”

Without that, FAIR becomes aspirational rather than actionable.

And we are not here to do an aspirational infrastructure project.

A moment of uncomfortable empathy

Here’s the part that surprised us.

In trying to understand why hosts wouldn’t step up, we found ourselves understanding, if not agreeing with, more of the structural tension Matt has been pointing at for years.

He has often argued that too many companies benefit from WordPress without contributing proportionally. That the burden of maintaining core infrastructure falls on too few shoulders. That the incentives are misaligned.

We still fundamentally disagree with how he has handled that frustration. The methods have caused damage. Public pressure campaigns and unilateral decisions are not the way to build trust.

But the underlying economic problem is real.

When hosts, some of whom generate significant revenue from WordPress, are unwilling to invest in a neutral stability layer, that tells you something about the ecosystem’s incentive structure.

And it tells you something about the limits of idealism.

The AI shift changes the battlefield

At the same time, the ground is shifting beneath all of us.

AI is transforming how websites are built, maintained, and even conceptualized. The barriers to publishing are falling again, not because of CMS innovation, but because AI systems can generate, deploy, and optimize digital experiences at unprecedented speed.

This changes the competitive landscape dramatically.

WordPress is no longer just competing with other CMS platforms. It’s competing with AI-native site builders, automated content systems, headless frameworks, and entirely new interaction models.

In that context, spending years fighting governance battles while the broader web evolves may simply be the wrong strategic focus.

The ecosystem needs to adapt to AI. To rethink workflows. To rethink value. To rethink what open source publishing means in a world where content can be generated infinitely and interfaces are conversational.

That is where energy is needed.

So: we are stepping away from FAIR

After much consideration, we’ve decided to stop working on bringing FAIR to the WordPress ecosystem.

Not because the idea was wrong.

Not because the governance concerns disappeared.

Not because everything is suddenly fine.

But because without hosting integration, including meaningful financial and structural support from key ecosystem players, the project is not viable.

Continuing would mean either building something underfunded and fragile which defeats the purpose or personally carrying a burden that should be shared.

Neither is responsible.

Ending our work on FAIR is not surrender. It is recognizing constraints. And we do so knowing that the work was not for nothing.

What we still believe

We still believe WordPress needs more transparent, accountable governance.

We still believe concentration of power creates systemic risk.

We still believe the community deserves better than polarization and fear.

But we also believe change cannot be forced by a handful of motivated individuals if the broader economic layer of the ecosystem refuses to participate.

Open source is not just code. It is a system of incentives.

And these incentives, right now, do not align for this kind of structural reform.

What’s next

For FAIR

At Cloudfest’s hackathon this year, we’ll work on making FAIR work with and for TYPO3. The TYPO3 community and technical leadership sees a lot of value in the system that was created, because the technical project actually delivered a good, technically solid, federated package management system.

In Europe, the concept of digital sovereignty is an incredibly hot topic, and an important part of that is being able to rely on package management servers. FAIR solves for that and we are proud to see that recognized by the TYPO3 community and to see them continue on the work we started.

For us

WordPress is entering a new phase, shaped not by internal governance debates, but by AI and platform shifts that will redefine publishing and the web itself.

That is where we want to focus our time and energy.

On helping WordPress businesses adapt.
On making plugins and tools smarter.
On ensuring independent publishers remain competitive.
On finding a path forward for Open Source in a world where a line of code has no value.
On building products that move the ecosystem forward, instead of fighting over its control structures.

The FAIR experiment clarified something important: if the ecosystem won’t fund neutrality, neutrality won’t materialize.

That’s a lesson worth learning.

A word of thanks

Many individuals have been with us along this road, and we’d be remiss not to thank them for their efforts. You know who you are, please know that you have our deepest thanks. We also owe thanks to the Linux Foundation. The team there has been incredibly helpful and stood by us as we took on this challenge.

We remain committed to WordPress. We remain critical where criticism is needed. We remain hopeful that governance can improve.

But for us both, FAIR ends here.

— Karim & Joost

Share Obsidian links that actually work everywhere

joost@joost.blog (Joost de Valk) — Wed, 25 Feb 2026 00:00:00 GMT

I’ve been playing with OpenClaw a lot, and I’m using Obsidian intensely with it. All my processes and even my OpenClaw’s core files live in my Obsidian vault. I have Obsidian on all machines I work on, so it’s very convenient. There was only one piece missing: a way to get links in WhatsApp from my OpenClaw that point back to Obsidian.

Obsidian links look gorgeous inside your vault, but drop one into WhatsApp or Telegram and you get a dead obsidian:// URI, useful only inside Obsidian itself. The whole point of obsid.net is to make those notes shareable in agent-human conversations: your bot can send a link in WhatsApp to obsid.net, and it redirects you straight into the right Obsidian note.

You don’t need to install anything for it to work. The obsidian:// URL handler just works, and obsid.net just redirects to it. It’s just a simple https://obsid.net/?vault=…&file=… link you can paste into WhatsApp, Slack, email, or any chat that refuses to run custom schemes.

Why care? Because in human + agent chats, the agent often is the one sharing notes to you. Codex, Claude, or any cron job can now output safe obsid.net URLs instead of raw obsidian:// URIs, and you can open them anywhere. obsid.net links also look like (no, are) normal HTTPS URLs, so they survive log archives, bookmarking, and every chat frontend you already use.

So, give obsid.net a try! Convert one of your old obsidian:// links, install the skill or helper for your agent, and then share the note inside WhatsApp or Telegram. No extra installs (well, except for the skills maybe), just a small redirect that keeps every Obsidian link tappable in your agent conversations.

Why healthy doubt beats AI confidence theater

joost@joost.blog (Joost de Valk) — Sat, 21 Feb 2026 00:00:00 GMT

AI will confidently sign off on anything. The question is whether you will too.

I’m a member of my old high school’s “AI council”. My kids go there now, so I have both my own experience and my kids’ to build on when I speak in that council. Recently they asked me to speak to a group of teachers there about AI, as they’re trying to help teachers navigate what AI means for schools. The subject they asked me to speak about was a logical choice for me: the implications of AI on the workforce.

This high school is what’s called a “Gymnasium” in the Netherlands. It’s the highest form of high school education we have where they teach you, besides all the normal subjects, ancient Latin and Greek. And because of that, those mythologies and philosophies too. As I was preparing, I realized this should be a blog post too.

The term “AI confidence theater” comes from Christina Aguilera’s piece on leadership, which is worth reading. She writes about executives performing certainty about AI while privately still figuring it out. That’s real. But this post is about a different layer of the problem: the AI itself.

The confidence problem

Ask Claude, Gemini, or ChatGPT anything and you’ll get a polished, articulate, supremely confident answer. Even when it’s wrong. Especially when it’s wrong.

Socrates built an entire philosophy around “I know that I know nothing.” AI does the opposite: it knows nothing about knowing nothing. It has no concept of its own ignorance. Every hallucination arrives dressed in the same crisp suit as every correct answer.

The philosopher Hannah Arendt drew a sharp line between thinking and knowing. Knowing is accumulating facts and producing answers. Thinking is the harder thing: examining those answers, sitting with uncertainty, refusing to stop at the surface. AI is a knowing machine. It has no capacity for the other thing.

This is the core problem we’re not talking about enough: we’ve built machines that perform certainty, and we’re raising a generation that’s learning to mistake that performance for truth.

The death of average

A passable 500-word essay now costs zero effort and ten seconds. A working function in most programming languages, same story. A first-draft marketing plan, a competitive analysis, a translation. All reduced to a button press. The floor hasn’t just risen. It’s been catapulted.

What this means practically: average output has no market value anymore. The “good enough” report, the “decent” first draft, the code that merely works. AI produces all of it faster, cheaper, and more consistently than any human can. If your only skill is producing average work, you’re competing against a machine that does it at near-zero marginal cost. You will lose.

But here’s what AI can’t do: it can’t tell you whether its output is actually good. It can’t feel that something is off. It can’t exercise judgment born from years of getting things wrong and learning why. That’s the human edge. Not production, but evaluation.

The junior paradox

This leads to what might be the most uncomfortable workforce question of the decade.

You become a senior by doing years of junior work. Aristotle argued in the Nicomachean Ethics that you learn craft and virtue through practice, by doing, repeatedly, until understanding becomes instinct. There’s no shortcut. You parse hundreds of sentences before Latin grammar becomes intuitive. You write thousands of lines of bad code before architecture clicks.

But if AI handles all the junior work, where do the future seniors come from? (I wrote about the flip side of this in The death of Code Copyright and the rise of the Architect: when execution becomes a commodity, what’s left is intent, direction, and logic. The Architect is the senior who made it through. The question is how we get there without the steps in between.)

I’m already seeing this play out. Developers who call themselves “senior” because they generate code at impressive speed, but who collapse the moment they hit a bug the model can’t resolve. They never put in the hours. They skipped the reps. They have the job title without the scar tissue.

And employers are catching on. Why hire a junior you need to train for three years when AI does their job today? It’s an obvious short-term decision and a slow-motion disaster for the industry. We’re removing the very path that made senior developers possible.

The only way through: the junior work still has to happen, but the bar for what constitutes “junior” has moved up dramatically. Entry-level now means you can do what AI does plus catch where it fails. That’s a fundamentally different starting point than five years ago.

Pilot or passenger?

There’s a useful mental model here: are you the pilot using autopilot to fly better, or the passenger who doesn’t know how to land the plane when the power goes out?

The autopilot metaphor is apt because real pilots don’t trust autopilot blindly. They understand every system it controls. They monitor. They override. They train constantly for the scenarios where automation fails, because it always eventually does.

Think of AI as a very capable intern (or 20 of them, in fact). Fast, eager, able to handle enormous volumes of routine work. But you’d never let an intern sign off on a contract, publish without review, or make an architectural decision. The person with skin in the game does that. The person who lies awake when it breaks.

The market is firing passengers and hiring pilots. It’s eliminating executors and desperately seeking directors: people with the autonomy and judgment to orchestrate AI output into something that actually works in context. The Architect, in other words.

The Descartes reflex

René Descartes might be history’s first debugger. His method: break any complex problem into the smallest possible pieces, verify each one, then rebuild.

This is exactly the skill AI is trying to make obsolete, and exactly the skill that matters most when working with AI.

AI always presents a smooth surface. The text reads well. The code compiles. The analysis looks thorough. But underneath, the logic might be fractured, the sources might be fabricated, the reasoning might be circular. Without the reflex to decompose and verify, to refuse to accept something as true just because it looks true, you’re at the mercy of the machine’s hallucinations.

I think of this as the Descartes Reflex: the trained instinct to stop, take apart, and verify before accepting. It’s not paranoia. It’s intellectual self-defense.

Classical education, interestingly, has been training exactly this reflex for centuries. Parsing a complex Latin sentence is decomposition. You can’t consume it whole. You have to break it apart, identify each grammatical relationship, and reconstruct meaning from the pieces. That’s not a dusty academic exercise. It’s the single most relevant cognitive skill for the AI age.

Sapere aude in 2026

In 1784, the German philosopher Immanuel Kant wrote a short essay with one central message: stop outsourcing your thinking to authority. He called it Sapere Aude: dare to think for yourself. He aimed it at a public too comfortable deferring to church and king.

Replace “church and king” with “algorithm and model” and the message is more urgent than ever.

Every time you accept an AI answer without interrogation, you’re choosing intellectual dependency. Every time you ask “why?” and dig into the reasoning, you’re exercising the kind of autonomy that no machine possesses.

This isn’t anti-AI. I use AI every day. I write code with it roughly 20x faster than I could alone. But every line that ships is still mine. If there’s a bug, it’s my bug. That’s not a limitation, it’s the point. You can use AI to outsource the work. You can’t outsource the responsibility.

What actually needs to change

If we take this seriously, a few things follow:

For educators: Stop grading the product (unless you know it was made in a controlled classroom with no phones in sight). Start interrogating the process. Asking “what’s your answer?” is a lost game, because AI will always have an answer. Ask instead: “Why this word and not that one? What was the alternative? How do you know this is right?” Be the Socratic gadfly. The students who can survive that interrogation are the ones who’ll thrive.

For managers and founders: Hire for judgment, not output volume. The person who can tell you why the AI’s answer is wrong is worth ten people who can generate AI answers quickly. Invest in humans who think, not humans who prompt.

For anyone entering the workforce: Your value isn’t in what you produce. AI produces more, faster, always. Your value is in what you understand. In the gap between “this looks right” and “this is right.” In the willingness to say “I don’t know yet” when the machine is already confidently wrong.

For all of us: Philosophy isn’t a luxury anymore. The skills it teaches (deconstructing arguments, questioning assumptions, distinguishing valid from sound reasoning, exercising judgment under uncertainty) aren’t nice-to-haves. They’re cognitive self-defense.

The uncomfortable truth

AI makes the average worker obsolete, but the excellent worker irreplaceable.

That’s not a comfortable message. It means the stakes are higher, the climb is steeper, and “good enough” is a dead end. But it also means that genuine human capability, the kind built through struggle, failure, and hard-won understanding, has never been more valuable.

The wind is in the sails. Humans still set the course. And when it goes wrong, you can’t blame the wind.

Building an Autonomous Feedback Agent

joost@joost.blog (Joost de Valk) — Sat, 14 Feb 2026 00:00:00 GMT

What if your users could submit a bug report and have a pull request ready for review before you’ve even seen the ticket? That’s what we built for Rondo Club, a sports club management app. It’s a fully autonomous feedback processing pipeline (in my case powered by Claude Code).

This post walks through the entire system: how feedback gets submitted, how an AI agent picks it up and writes code, how PRs get reviewed automatically, and how it all runs unattended on a Mac Mini (but could run anywhere).

The idea
How it works
Stage 1: Feedback submission
Stage 2: Approval
Stage 3: The feedback agent
Stage 4: Automated code review
Stage 5: Merge and deploy
Stage 6: Fixing errors from the server
Stage 7: Code optimization
The conversation thread
The things I learned
- What works well
- What to watch for
The stack
Running it yourself
Tried this yourself? Have feedback?

The idea

Rondo Club is a WordPress-based application with a React frontend, used by my local football club to manage members, teams, and committees. As a solo developer, I wanted to shorten the loop between “user reports a problem” and “fix deployed to production”, without me being the bottleneck for every small bug or feature request.

The goal: users submit feedback in the app, I approve it with a click. Then an AI agent processes it, creates a PR, gets it reviewed, and merges it, all without further human intervention for straightforward changes.

How it works

The system has six stages:

Submission — Users submit feedback through an in-app form.
Approval — An admin reviews and approves the feedback item.
Processing — A scheduled script picks up approved feedback and sends it to Claude Code.
Review — The resulting PR gets an automated code review.
Resolution — Clean PRs are merged and deployed automatically.
Optimization — When the feedback queue is empty, the agent reviews existing code for improvements.

Let’s walk through each stage.

Stage 1: Feedback submission

Every logged-in user sees a feedback button in the app’s navigation. Clicking it opens a modal where they can submit either a bug report or a feature request.

For bugs, the form captures:

Title and description
Steps to reproduce
Expected vs. actual behavior
Optional screenshot attachments (drag-and-drop)
Optional system info (browser, app version, current URL)

For feature requests:

Title and description
Use case

Users also select which project the feedback is for, as the system supports multiple repos (the main app, a sync service, and a marketing website).

On the backend, feedback items are stored as a WordPress custom post type with metadata for status, type, priority, and project. A REST API endpoints handles CRUD operations and supports filtering by status and type.

New feedback items start with status new.

Stage 2: Approval

This is the only human gate in the process, and it’s an important one. An admin reviews each feedback item and sets it to approved when it’s ready for the agent to pick up. This prevents the agent from acting on vague, duplicate, or potentially malicious input. Without this step, any logged-in user could effectively instruct an AI agent to write and merge code into your codebase.

Stage 3: The feedback agent

The heart of the system is a bash script that runs on a Mac Mini via launchd, scheduled every 5 minutes. It operates in loop mode: fetch the oldest approved item, process it, repeat until the queue is empty.

Fetching work

Each iteration follows a priority order:

In-review PRs first — Check if any existing PRs have Copilot review feedback that needs addressing. Finish what’s already started before taking on new work.
Approved feedback — Pick up the oldest approved item from the API.
Optimization — Only when nothing else needs doing, review code for improvements.

Running Claude Code

When the script picks up a feedback item, it:

Sets the item’s status to in_progress (so no other run picks it up)
Formats the feedback into a structured prompt with all context: description, steps to reproduce, conversation history, and system instructions
Passes it to Claude Code with --print --dangerously-skip-permissions
Monitors the process with a 10-minute timeout, logging progress every 2 minutes

The prompt ends with an agent instruction file (.claude/agent-prompt.md) that tells Claude exactly what to do:

Create a feedback/{id}-{slug} branch
Analyze the feedback and make changes
Run npm run build to verify the frontend compiles
Commit, push, and create a PR with the feedback ID in the title
Output a structured status (IN_REVIEW, NEEDS_INFO, or DECLINED)

Status handling

When Claude finishes, the script parses its output for a status line:

IN_REVIEW — A PR was created. The script extracts the PR URL, stores it as metadata on the feedback item, and requests a Copilot code review.
NEEDS_INFO — Claude couldn’t resolve the issue autonomously. The script posts Claude’s question as a comment on the feedback thread, visible to the user in the app.
DECLINED — The feedback was evaluated and determined not actionable.

If Claude fails or times out, the status resets to approved so it re-enters the queue.

Safety features

The script includes several safety mechanisms:

Lock file prevents concurrent runs (launchd might trigger while a previous run is still going)
Signal handlers catch SIGTERM/SIGINT and clean up — resetting feedback status and checking out main
Timeout enforcement kills hung Claude processes after 10 minutes
Crash recovery resets feedback status if the script exits unexpectedly
Git cleanup returns to main and removes merged branches after each item

Stage 4: Automated code review

After Claude creates a PR, the script requests a review from GitHub Copilot. On the next run, the PR review processor checks for responses:

Clean review (no inline comments) — The PR is safe to merge. Proceed directly to Stage 4.
Review with comments — The script checks out the PR branch, formats Copilot’s inline comments into a prompt, and runs Claude again with a review-fix instruction set (.claude/review-fix-prompt.md).

The review-fix prompt tells Claude to:

Fix real bugs, consistency issues, and safety problems
Skip stylistic nitpicks, test requests, and subjective suggestions
Output whether the PR is SAFE_TO_MERGE after fixes

If Claude determines it’s safe to merge after fixing review comments, the PR proceeds to automatic merge (and if it fails, which can happen because other PRs have been merged since etc, Claude fixes the merge conflict). Otherwise, it gets assigned to the human developer for manual review.

Stage 5: Merge and deploy

When a PR is deemed safe — either by a clean Copilot review or by Claude’s post-fix assessment — the script:

Squash-merges the PR via gh pr merge --squash --delete-branch
Pulls main with the merged changes
Runs the deploy script, which rsync’s the theme to production and clears caches
Updates the feedback item status to resolved

The feedback ID is extracted from the branch name (feedback/1234-slug → 1234) to automatically close the right item.

Stage 6: Fixing errors from the server

Once the feedback queue is empty and the reviewed PRs are handled, the agent starts looking at the server’s PHP error logs. If it finds errors it can fix, it’ll run the same flow for them as it does for feedback items.

Stage 7: Code optimization

When the feedback queue is empty, all the PRs are handled and there are no errors on the server that need work, the agent enters optimization mode. It systematically works through every source file across all projects, reviewing each one for simplification opportunities: dead code removal, DRY violations, unnecessary complexity, performance wins. If it finds confident improvements, it creates an optimize/ PR. If not, it moves on.

To avoid wasting API calls re-reviewing unchanged code, the tracker stores the git commit hash that last touched each file. On subsequent runs, it compares the stored hash against the current one — if they match, the file hasn’t changed and gets skipped. Only files with new commits are re-reviewed.

A tracker file prevents reviewing the same file twice and enforces daily limits (25 optimization runs, 10 PRs per day) to keep costs reasonable.

The conversation thread

One of the more useful features is the back-and-forth capability. When Claude responds with NEEDS_INFO, its question gets posted as a comment on the feedback item. The user sees this in the app’s feedback detail page and can reply.

On the next processing run, the agent picks up the item again — this time with the full conversation history included in the prompt. Claude sees the original feedback plus all follow-up exchanges, giving it the context to resolve the issue.

This means some feedback items go through multiple rounds:

User submits bug report
Admin approves
Agent asks clarifying question
User responds
Admin approves again (to prevent injection of vile stuff)
Agent fixes the bug and creates a PR

The things I learned

What works well

Small, focused changes are reliable. Claude handles targeted bug fixes and UI tweaks consistently well. The structured feedback format (steps to reproduce, expected/actual behavior) gives it enough context to work autonomously.
The priority system matters. Finishing in-review PRs before starting new work prevents a pile-up of half-done items.
Timeouts are essential. Claude occasionally gets stuck in an idle state. The 10-minute timeout with progress logging catches these cases automatically.
The human gate is important. Having an admin approve feedback before it enters the queue filters out vague or duplicate reports that would waste agent time.

What to watch for

Claude sometimes over-engineers. The agent prompt explicitly says “keep changes minimal and focused” — without this, Claude tends to refactor surrounding code while fixing a bug.
Process monitoring needs visibility. Early on, stuck processes were invisible. Adding progress logging every 2 minutes and structured log output made it much easier to diagnose issues remotely.
Git state management is tricky. The script needs to handle dirty working directories, failed checkouts, and stale branches defensively. Every Claude session might leave the repo in an unexpected state.

The stack

ComponentTechnologyFeedback storageWordPress custom post type + REST APIFeedback UIReact form with drag-and-drop attachmentsProcessing scriptBash (get-feedback.sh)AI agentClaude CodeCode reviewGitHub CopilotPR managementGitHub CLI (gh)SchedulingmacOS launchd (every 5 minutes)Deploymentrsync via bash scriptHostingMac Mini (agent) + WordPress## Running it yourself

The core pattern is transferable to any project:

Build a feedback intake — could be a form, a Slack bot, a GitHub issue template, whatever captures structured input.
Write an agent prompt — be very specific about what the agent should and shouldn’t do. Include rules about branch naming, commit messages, and PR format. Tell it to keep changes minimal.
Wrap Claude Code in a monitoring script — handle timeouts, cleanup, and status tracking. Don’t trust that the process will always complete cleanly.
Add automated review — a second pass catches issues the agent might introduce. This could be Copilot, another Claude session, or a linter.
Automate the merge/deploy for safe changes — but keep a human fallback for anything complex.

The full source is in the Rondo Club repository — the agent script at bin/get-feedback.sh and the prompts in .claude/.

Tried this yourself? Have feedback?

Let me know in the comments!

Great minds think alike? My WordPress take on Markdown for Agents

joost@joost.blog (Joost de Valk) — Thu, 12 Feb 2026 00:00:00 GMT

Today, Cloudflare announced Markdown for Agents, a feature that automatically converts HTML to markdown at the edge when AI agents request it. Reading their announcement felt like looking in a mirror. I’ve been building the exact same concept, even up to including frontmatter with metadata, independently, as a WordPress plugin called Markdown Alternate (which has been running on this blog for almost two weeks).

The core insight is identical: the web was built for humans, but AI agents are now a significant — and growing — consumer of web content. Feeding raw HTML to an LLM is wasteful. Cloudflare cites an 80% token reduction when converting their own blog post from HTML to markdown. That matches what I’ve seen.

Both solutions use the same mechanism: HTTP content negotiation via the Accept: text/markdown header. When an agent sends that header, it gets markdown back instead of HTML. Tools like Claude Code and OpenCode already send this header. The infrastructure is ready, the content just needs to follow.

Cloudflare’s solution operates at the CDN edge. It’s broad: flip a toggle in your dashboard, and any page on your zone can be served as markdown. The conversion is generic, Cloudflare doesn’t know whether your page is a blog post, a product page, or a documentation article. It converts whatever HTML it sees.

What `markdown-alternate` does

My plugin operates at the application level, inside WordPress. That makes it narrower in scope but significantly deeper in what it can do. It offers:

Dedicated `.md` URLs.

Markdown Alternate gives every post and page its own .md endpoint. Instead of only relying on content negotiation, you can simply request /my-post.md and get markdown back. These URLs are bookmarkable, independently cacheable, and intuitive, just like .json or .xml endpoints. Cloudflare only supports the Accept header approach. Of course they do add rel="canonical" HTTP headers back to the HTML source.

Discoverability.

The plugin adds <link rel="alternate" type="text/markdown"> tags to every page’s HTML head. An agent visiting the HTML version can programmatically discover that a markdown version exists. Cloudflare doesn’t offer this, agents have to know to ask for markdown upfront. It might actually be a good addition for their solution.

Richer metadata.

Cloudflare’s frontmatter includes title, description, and image. Because my plugin lives inside WordPress, it has access to the full post object. The frontmatter includes the title, publication date, author, featured image, and categories and tags, each with their own .md URLs, so agents can navigate related content.

WordPress-aware content processing.

The plugin knows what type of content it’s processing. It strips syntax highlighting markup that plugins like Highlight.js inject into code blocks, preserving clean code with language hints intact. A generic HTML-to-markdown converter can’t easily know about this.

“Stealing” the good ideas

Cloudflare’s announcement included one feature I immediately wanted: the X-Markdown-Tokens header. It returns an estimated token count for the markdown response, letting agents plan their context window usage or chunking strategy before processing the content.

I added it to Markdown Alternate within minutes of reading their post. The implementation is simple, strlen / 4 gives a reasonable estimate for English text, but the idea of exposing this as a header is clever. It costs nothing to produce and gives agents useful information they’d otherwise have to calculate themselves.

Complementary, not competing

These two approaches aren’t in conflict. A WordPress site could use Markdown Alternate for rich, WordPress-aware markdown with dedicated URLs and full metadata, while Cloudflare’s feature provides a baseline for every other site on their network. The plugin gives you control and depth; Cloudflare gives you breadth and zero effort.

The bigger picture is that the web is adapting to a new kind of consumer. HTML isn’t going away, but the expectation that every consumer of web content is a browser is. Markdown is emerging as the lingua franca for AI systems, and the sooner publishers start serving it natively, the better their content will perform in an agent-driven world.

Markdown Alternate is available on GitHub.

The silence is deafening: Google’s “agentic” future leaves the WordPress economy behind

joost@joost.blog (Joost de Valk) — Mon, 12 Jan 2026 00:00:00 GMT

Google just announced a massive shift in how the internet shops, and the biggest platform on the web wasn’t even in the room.

If you haven’t seen it yet, Google recently dropped a bombshell announcement regarding the future of online retail. They are rolling out a new “Universal Commerce Protocol” (UCP) and a suite of “agentic commerce” tools. In plain English? They are building a standard language for AI agents to browse, negotiate, and buy products on behalf of humans.

You can read the full announcement here: New tech and tools for retailers to succeed in an agentic shopping era.

The list of launch partners is a who’s who of modern commerce: Shopify, BigCommerce, Etsy, Wayfair, Walmart, and Target. Financial giants like Stripe, Visa, and Adyen are backing it.

But if you look closely at that list, you’ll notice a gaping hole.

WordPress and WooCommerce are nowhere to be found.

The representation gap

This omission exposes a dangerous structural flaw in our community.

WooCommerce isn’t just a plugin; it is an economy. It feeds a massive ecosystem of hosting companies (for example, GoDaddy, WP Engine, SiteGround & Kinsta), premium plugin developers (like StellarWP, Elementor, and Awesome Motive), and thousands of agencies.

These companies have the most to lose. If WooCommerce slides into irrelevance because it can’t “speak” to Google’s new AI agents, their revenue streams dry up. Yet, none of these companies have a seat at the table.

When Google calls a meeting to define the future of the web, they (of course) don’t invite a swath of individual hosting CEOs or plugin developers. They invite the platform owner. For Shopify, that’s Tobi Lütke. For WooCommerce? It should be Automattic.

This falls squarely on Automattic’s shoulders. As the steward of the brand and the “owner” of WooCommerce, they are the only entity with the perceived authority to represent this massive ecosystem in high-level enterprise negotiations. If they aren’t in the room, we aren’t in the room.

The “open web” liability

For years, we in the WordPress community have taken pride in being the “open web”. We own our data. We don’t pay monthly platform fees to a landlord. But this announcement signals a terrifying reality: Being “open” doesn’t matter if the new gatekeepers can’t speak your language.

Google’s new protocol is designed to standardize the chaotic world of checkout and product data so that AI (specifically Gemini) can read it instantly. By partnering directly with Shopify and BigCommerce, Google is effectively creating a “fast lane” for those merchants.

If you are on Shopify, your products will soon be natively understandable by Google’s AI agents. A user asks Gemini for “a good running shoe,” and the AI can theoretically find it, vet it, and even initiate the checkout without the user ever wrestling with a clunky website.
If you are on WooCommerce, right now, it looks like you are stuck in the slow lane, waiting for a plugin developer to build a bridge to this new world.

The downstream disaster

The danger isn’t just that a few store owners lose sales. The danger is systemic.

If WooCommerce becomes the “Linux of ecommerce”, powerful, beloved by geeks, but invisible to the modern AI consumer, merchants will leave. And when merchants leave, they don’t just cancel a free plugin. They cancel their $50, $100, $500/month hosting plan. They cancel their plugin subscriptions. They stop hiring WP agencies.

The hosting and plugin companies that profit from the WordPress ecosystem are currently relying on Automattic to keep the platform viable at the enterprise level. But if Automattic is too distracted by legal battles or simply not prioritized by big tech, the entire downstream economy suffers.

Is there hope? (And why we need FAIR)

It’s not all doom and gloom, yet. But waiting for Automattic to fix this might be a strategy we can no longer afford.

This is exactly why initiatives like FAIR are becoming critical. We need a neutral, decentralized body that can represent the business interests of the WordPress ecosystem (the companies I mentioned above and the many others out there) without being tied to a single company’s roadmap.

If Automattic won’t sit, or doesn’t get invited to sit, at the table with Google to implement the Universal Commerce Protocol, then the hosting and plugin companies need to do it themselves. FAIR could be the vehicle to standardize our data structures, giving us a collective voice that even Google can’t (and wouldn’t even want to) ignore.

If we don’t figure out how to unify our voice, either through Automattic or through a new coalition, we won’t just be independent. We’ll be irrelevant.

The Tailwind paradox: the high price of “enough”

joost@joost.blog (Joost de Valk) — Sun, 11 Jan 2026 00:00:00 GMT

The news hit the developer community like a cold bucket of water. Adam Wathan, the creator of Tailwind CSS, recently announced that he had to lay off 75% of his engineering team, cutting the staff from four engineers down to just one.

The numbers he shared were staggering. Despite Tailwind being more popular than ever (with 75 million downloads a month), traffic to their documentation is down 40% since 2023. Worse, revenue is down nearly 80%.

The culprit? AI.

As Adam explained, developers are no longer visiting the docs to learn how to center a div; they are asking ChatGPT or Claude. Since the documentation site was the primary funnel for their commercial products (Tailwind UI), the business collapsed even as the tool’s usage skyrocketed.

Dries Buytaert recently published a thoughtful reaction to this, titled AI is a business model stress test. He argues that AI is forcing a shift from “Specification” (code/docs) to “Operation” (hosting/services).

He is absolutely right. But as I read it, I kept thinking that multiple things can be true at once, with #3 being the most tragic in my eyes:

AI unfairly captured value it didn’t pay for.
Tailwind’s pricing model was a ticking time bomb regardless of AI.
Their desire to be “good” open-source citizens is exactly what left them vulnerable.

I want Tailwind to win

Before I get into the economics, let me be clear: I love Tailwind. I use it constantly. I have personally purchased their lifetime “all-access” licenses multiple times for different companies, including Yoast and Emilia Capital.

I am the ideal customer. I bought the product because it provides immense utility, and I wanted to support the creators. It’s painful to see a team that has built something so useful struggle.

The great value heist

Dries points out a harsh reality in his piece: AI broke the funnel.

For years, Tailwind’s business model was a perfect loop: Developers searched for answers → landed on Tailwind docs → saw the beautiful Tailwind UI ads → bought a license.

AI severed that link. LLMs were trained on Tailwind’s documentation, examples, and community discussions. Now, when a developer asks, “How do I center a div in Tailwind?” the AI provides the answer instantly.

The AI creates value for the user by using Tailwind’s intellectual property, but it captures 100% of the attention. Tailwind Labs gets $0 and 0 clicks. It is, effectively, an extraction of value without compensation. When I was discussing this with Jono, he mentioned, I think correctly, that the same risks apply to any product/service where the primary function is to abstract away the complexity of primitives.

But even if we solved that problem, even if AI didn’t exist, Tailwind was facing a mathematical wall.

The “forever” problem in a finite market

Tailwind UI launched with a promise that developers (myself included) love: Pay once, own it forever.

In the beginning, this looks like a rocket ship. You have a wide-open Total Addressable Market (TAM). Every sale is a massive injection of cash upfront. It feels like valid, sustainable growth.

But financially, a lifetime license is a liability. You collect all the revenue you will ever get from a customer on Day 1, but you commit to supporting them and updating the product forever.

There is a finite number of frontend developers in the world. There is a smaller number who use Tailwind, and a smaller number still willing to pay $299 for UI components.

Lifetime pricing was never going to be a good idea in any finite market.

The tragedy of “enough”

This is the part that breaks my heart. Knowing the open-source ethos, the creators almost certainly looked at their early success and thought, “This is enough.”

They didn’t want to be the “greedy” corporation extracting a monthly toll from developers. They wanted to be the good guys. They likely looked at their bank balance, saw millions from the initial rush of lifetime sales, and calculated that they could run the company indefinitely on that pile + a trickle of new sales.

The irony is brutal: had they been more “greedy,” they would be safe.

If the Tailwind team had ignored its modest instincts and implemented a standard SaaS subscription model, it would currently have a recurring revenue floor.

With subscriptions, a 40% drop in traffic hurts growth, but the existing millions of users keep the lights on.
With lifetime deals, the existing millions of users are financially irrelevant. The company relies entirely on new customers to survive.

By trying to be fair to their community, they stripped their business of the resilience it needed to survive a market shift.

The lesson

Dries is right that the market is shifting from “Specification” to “Operation”. He’s also right that AI is a business model stress-test. Maybe even more so than he puts in his words.

Because for the next generation of founders, the lesson is starker. “Enough” is a concept that applies to personal finance, not corporate operations. A company needs flow, not just a pile.

I want Tailwind to be around for another ten years. But for that to happen, they (and future founders) need to realize that sustainable revenue models aren’t about greed; they are about survival.

From installation to integration: Making plugins “agent-ready”

joost@joost.blog (Joost de Valk) — Wed, 24 Dec 2025 00:00:00 GMT

In my last post, I discussed why a design system is the “visual rail” AI needs. But the “Architect” I’ve been describing doesn’t just care about how a site looks; they care about how it functions.

The problem today is that even the smartest AI is often a “clueless” collaborator. You ask a coding assistant to “add a contact form to the sidebar,” and instead of using the powerful form plugin you already have installed, it starts writing a custom database table and a raw PHP mail() function from scratch. It’s not being “creative”, it’s just unaware that the tools for the job are already sitting in your site’s plugins folder.

To fix this, we have to stop treating plugins like “black boxes” and start treating them as self-advertising modules.

The communication gap is (mostly) solved

The good news is that we already have the “telephone line” for this conversation. The WordPress AI team recently introduced the Abilities API and the MCP adapter.

The Abilities API creates a centralized registry where plugins can formally register their functionalities with well-defined schemas. This means that, once installed, a plugin can “hand a menu” to an AI. But for an Architect to plan a site, they need to see that menu before the plugin is ever installed.

The discovery layer: `AGENTS.md`

The emerging AGENTS.md standard is the right choice for our ecosystem. It isn’t just a text file for a crawler; it’s a briefing document for the AI agents that will be doing the building.

By placing an AGENTS.md file in the root of a plugin, we provide a human-and-AI-readable introduction that links directly to a more technical abilities.yml. This YAML file acts as a static export of what the plugin’s registered abilities do, following the same schema as the Abilities API:

abilities:
  - id: "my-seo-plugin/analyze-content-seo"
    label: "Analyze content SEO"
    description: "Analyzes post content for SEO improvements."
    input_schema:
      type: "object"
      properties:
        post_id:
          type: "integer"
          description: "The post identifier."
          required: true
    output_schema:
      type: "number"
      description: "The SEO score in percentage."

With this pairing, the Architect can scan a dozen plugins on a remote repository and know exactly which ones fit the site’s requirements, down to the exact data types required, without downloading a single byte of PHP.

Discovery via the FAIR.pm supply chain

This vision requires a trusted way to query and distribute this information. This is where FAIR.pm comes in.

If the FAIR network indexes these AGENTS.md and abilities.yml files, the global supply chain for WordPress becomes machine-readable. An Architect can query FAIR to ask: “Find me a trusted package that exposes an ability to analyze a post’s SEO, preferably with an output schema of ‘percentage’.” FAIR provides both the trust and the map, while the AGENTS.md ecosystem provides the technical contract.

Of course, this could be done on the WordPress.org plugin directory too, and it would be nice if it did, but that currently does not include any commercial plugins. FAIR is planning on allowing for all plugins from every host, which would make this a lot more inclusive of every plugin out there.

The “agent-first” developer experience (AX)

For twenty years, plugin developers have obsessed over the user experience (UX). In the “Rise of the Architect” era, developers must at least also prioritize the agent experience (AX).

If we don’t create a standard like this, plugins may provide a beautiful settings page, but they are invisible to the modern architect. We have to find a way for plugins to be considered in AI assembly/development processes before they’re installed.

Conclusion: legibility is the new portability

Plugins aren’t going away, but they are changing from being “destination apps” inside the WordPress admin to being “service providers” for an AI-driven assembly process.

By adopting a shared manifest of capabilities and plugging into decentralized networks like FAIR.pm, we ensure that the WordPress ecosystem remains a coherent library of tools, even if they’re spread across many different repositories. If we give the AI a map of our abilities, it can finally stop reinventing the wheel and start helping us build the house we want using already existing parts.

Vibe coding is a trap: why WordPress needs a design system NOW

joost@joost.blog (Joost de Valk) — Tue, 23 Dec 2025 00:00:00 GMT

In my last post, I argued that WordPress needs to become a “Base AI”: a structured foundation that AI can understand and build upon. Before that, I wrote about The Rise of the Architect, the person who will shift from writing code to orchestrating systems.

But there is a massive obstacle standing in the way of this future: WordPress is currently a “system” without a shared language.

We are seeing this play out in real-time with tools like Telex. It’s a glimpse into “Vibe Coding”, the ability to describe a UI and watch the AI manifest it. It’s seductive. It’s fun. And if we aren’t careful, it’s going to lead to a maintenance nightmare that makes the “spaghetti code” of the 2000s look like a masterpiece.

The seduction of the “vibe”

Tools like Telex allow you to bypass the traditional development process. You want a pricing table with a specific “bounce”? You describe it, the AI generates the React and CSS, and it appears.

To a “Site Builder” persona, this might feel like magic. To an Architect, this feels like a ticking time bomb.

The problem with “vibe-coded” blocks is that they are disposable code. They are isolated islands of logic and style. When an AI generates 20 different blocks and plugins for your site based on 20 different prompts, you haven’t built a website; you’ve built 20 independent technical debt projects.

If you decide to change your brand’s primary blue or adjust your site’s spacing logic, you can’t just update a central variable. You have to “re-vibe” the entire site.

The missing foundation: a design system

This brings me back to a point I’ve made before: WordPress desperately needs a unified design system.

For years, the lack of a standardized Admin UI has hampered WordPress. Every plugin developer had to invent their own buttons, toggles, and layout logic. This was already a bad user experience for humans, but it’s a catastrophic “architectural” failure for AI.

An AI doesn’t need to be “creative” with your layout; it needs constraints.

Without a design system: AI is like a toddler with a bucket of clay. It can mold anything, but nothing it makes will ever fit perfectly with the next thing.
With a design system: AI is an Architect with a box of Legos. Everything snaps together because the rules, the tokens for spacing, color, and typography, are pre-defined.

System-first vs. prompt-first

The industry is currently obsessed with “prompt-first” development (Telex). But the “Architect” I described in my earlier posts knows that the future must be “system-first.”

In a system-first world, the AI doesn’t write the CSS for a button. Instead, the AI selects the “Primary button” component from the WordPress Design System and configures it. The “Base AI” ensures that the output is predictable, maintainable, and, most importantly, updatable.

If WordPress remains a collection of “vibes” rather than a rigid system of components, we are effectively building a web of disposable components. We are trading the long-term health of our sites for the short-term high of watching an AI generate a pretty block in ten seconds.

The choice

We are at a fork in the road.

We can follow the path of unstructured generation, where AI fills WordPress with an endless stream of unmaintainable, bespoke code. Or, we can finally do the hard work of building a robust design system that provides the “rails” the AI needs. Of course, we’d then also have to ensure plugins and themes use that design system, but I think that with a good carrot approach (in which we promote those that use the design system), we can go a long way.

If we want the “Rise of the Architect” to be a success, we need to stop asking AI to be a painter and start asking it to be an assembler. But first, we have to give it the right base parts.

The generalization tax: why WordPress is still the smart architectural base

joost@joost.blog (Joost de Valk) — Mon, 22 Dec 2025 00:00:00 GMT

In my previous post, I discussed the demise of code copyright and mentioned what Dries referred to as the generalization tax. This had me thinking more about what that means for WordPress and investing in its ecosystem.

For twenty years, the winning strategy in software was to build the “base layer”. WordPress won this race by becoming the engine for 40%+ of the web. It succeeded by being everything to everyone.

But as AI code generation matures, a new marketing threat is emerging. It is the promise of the “Bespoke Engine”. Why deal with the overhead of WordPress when an AI can generate a lean, custom-built solution just for your specific needs?

The two sides of the bet: Bull vs. Bear

In the world of investing, your outlook on a market is usually described as being either “Bullish” or “Bearish.”

The Bear case (pessimistic): A “bear” believes a market or product is headed for a decline. The Bear Case for WordPress is that it has become too heavy. It pays a massive “Generalization Tax” where hundreds of developers spend thousands of hours writing code to handle edge cases for millions of users. The Bear argues that AI will allow developers to bypass this bloat entirely by building tiny, custom engines from scratch.
The Bull case (optimistic): A “bull” believes a market will grow or remain dominant. The Bull Case for WordPress is that its massive scale is its greatest security. Because it is used by 40% of the web, its core logic is battle-tested in a way that no custom AI engine could ever be.

Why I am a proponent of the bull case

I lean toward the Bull case, but not because I want to protect the old way of doing things. I believe WordPress is the superior long-term option because it is the only way to prevent “AI Slop”.

Using WordPress (along with robust tools like ACF) as a base for custom applications is a sophisticated architectural choice. If you let an AI generate a “lean, custom engine” from scratch today, you are creating a massive maintenance liability for tomorrow. Without a standardized base, that AI-generated code will eventually become “slop”: unpatchable, insecure, and disconnected from the global web standards.

The WordPress bull case is actually about risk mitigation:

The standardized logic: WordPress handles the “boring” but critical logic (Users, Auth, Database, Security) that has been battle-tested over two decades.
The battle-tested infrastructure: By using a generalized base for basic needs, you ensure that your custom features are built on a foundation that receives global security updates.
The architect’s canvas: The modern architect might use AI to build high-value custom plugins on top of this base, rather than wasting time reinventing the wheel.

The marketing threat: “architecture” vs. “bespoke”

Even if the “bespoke” promise is technically fragile, the WordPress community needs a coherent answer to this marketing narrative. Right now, it doesn’t have one.

The industry is currently being told that “custom is faster.” We need to tell the story of the modern architect. In this story, the architect doesn’t spend time on the “bricks” of a login screen or a database schema. They use WordPress as their “base” to ensure their vision doesn’t turn into a technical debt nightmare three years down the road.

Conclusion: investing in maintainable scale

From an investment perspective, the value isn’t just in the market share. It is in a maintainable scale. The risk of the “AI-bespoke” movement is that it creates a fragmented web of one-off systems that will fail the moment the AI model or the underlying language evolves.

The “Generalization Tax” is real, but it is a price worth paying for a foundation that won’t crumble. If WordPress wants to win the next decade, it has to move from selling “the website builder” to selling “the foundation for AI-driven website architecture”.

Read my next post to understand why I think vibe coding is a trap.

The death of Code Copyright (and the rise of the Architect)

joost@joost.blog (Joost de Valk) — Sun, 21 Dec 2025 00:00:00 GMT

We are witnessing a strange paradox in software development. Thanks to AI code generation, more open source code is being created today than ever before. Yet, simultaneously, the value of a single line of code has never been lower.

The commons (the vast library of open source projects that allowed AI models to learn how to code in the first place) is now being disrupted by the very technology it enabled.

The erosion of the license

In the traditional open source model, licenses like the GPL or MIT protect the expression of an idea. If you used someone’s code, you followed their rules. But AI has created a massive loophole.

When code can be rebuilt from scratch in seconds, the copyright on the specific syntax does not matter much. If a developer uses AI to replicate your tool’s exact functionality without copying a single line of your original source, they haven’t “stolen” your code in a legal sense. They have “developed it themselves”.

This makes building small utility pieces of software less interesting as a way of making money. These tools are now so easily copied that attribution and protection are nearly impossible to enforce. You do not need to attribute if you did not use the code, even if you fully stole the idea and the functionality.

The Architect’s toolkit: intent, direction & logic

If the code itself isn’t worth much, what is? We are seeing the value move upstream, away from the manual labor of coding and toward the high-level design of the solution.

In the past, a developer was often a glorified bricklayer. Success was found in the manual labor of laying down stable, reusable bricks of code. But when AI can manufacture and lay those bricks instantly, the value of the bricklayer vanishes.

The value now lies in the Architect. This new role isn’t about writing syntax. It is about providing the three things AI cannot yet originate:

Intent: Defining the “why.” Identifying exactly what problem needs solving and why it matters to the user.
Direction: Steering the solution. Knowing which path to take when AI offers infinite variations and ensuring the project stays true to its goal.
Logic: The structural “how.” Designing the underlying flow and logic that ensures separate systems work together as a cohesive whole.

The irony is that these architectural components (the soul of the software) cannot be protected by open source licenses or copyright. We are moving into an era where intent is everything, and execution is a commodity.

The value of the public domain

While the legal protection of a license is fading, the fact that code lives in the public domain matters more than ever. This vast, shared library of logic is what allows us all to develop faster. We are no longer starting from zero. We are starting from a collective baseline of human knowledge that AI has made instantly accessible.

By removing the friction of syntax and boilerplate, developers are free to have much grander ideas. We can now build complete, complex solutions much faster than we could five years ago. We are no longer limited by how many bricks we can lay in a day. We are only limited by the scale of our vision.

The “Dries” model: code as a blueprint

A perfect example of this shift comes from Dries Buytaert, the founder of Drupal. In a recent blog post, Dries discussed “Adaptable Modules.” These are site-specific pieces of code that are not meant to be installed, but rather used as a starting point for AI to reshape.

Dries is sharing his architectural choices without worrying about what he calls the “generalization tax,” which is the massive effort required to make code work for everyone. He provides the blueprint (his logic and direction) and lets AI handle the manual labor of adapting it to a specific site.

In this new world, the win is not in protecting the bricks or ensuring your name is on every derivative work. The win is in the proliferation of the solution. Dries might not even care about getting recognition when his code is repurposed, but that is a luxury of the established.

Vision is the new competitive edge

For the developer just starting out today, the landscape has shifted permanently. The protection once offered by copyright is dissolving into a sea of AI-generated syntax.

The rise of the architect means that your value no longer lives in the files you commit to GitHub. It lives in your ability to define intent, maintain direction, and master the logic of a solution.

You can no longer compete on the bricks. To survive and thrive in this new era, you have to compete on vision.

Read my next post about the generalization tax, and why I still think WordPress is a great base for AI.

A new path forward for WordPress, and for the open web

joost@joost.blog (Joost de Valk) — Fri, 06 Jun 2025 00:00:00 GMT

In December, I wrote about the state of leadership in the WordPress ecosystem and what should be on WordPress's roadmap. I shared how too much power rests with one person, and how the lack of transparent governance puts contributors and businesses alike in difficult positions. That post ended with a call: we need to lead. That wasn’t rhetorical. It was a pivot.

Since then, a lot has happened. Today, I want to share what came next and where it’s going. This is the story of FAIR, a project I first named in that same blog post: Federated and Independent Repositories. At the time, it was just a placeholder for an idea. Now, it’s real, and it’s running.

The moment the ecosystem shifted

In October, Sarah Savage published a post introducing AspirePress, a community-run mirror of the WordPress.org plugin and theme repositories. That post was a spark. It showed what was technically possible, and voiced what many of us had been saying in smaller rooms for years: the ecosystem needed options.

Then, in early December, twenty core contributors wrote an open letter calling for governance reform in WordPress. These weren’t newcomers. They were committers, team leads, people who had spent years helping build the platform. Their message added weight to the concerns many had been feeling in private.

Shortly after, Karim Marucchi and I published our thoughts. We proposed a dual-track forward: one that addressed technical issues like centralization, update bottlenecks, and discoverability, and one that introduced a different approach to governance, governance that would be transparent, accountable, and neutral.

That combination turned out to be a catalyst. Even more so after Matt blogged about it in a sarcastic post on WordPress.org. People began reaching out, and conversations happening in parallel started to overlap.

From parallel threads to shared momentum

Several groups had been working on related problems, each bringing different perspectives, needs, and capabilities. Rather than announce a new umbrella, we focused on connecting the dots between these parallel efforts. We’re not naming everyone in that alignment publicly yet, but it’s safe to say: this is a group of groups, and that’s its strength.

What followed were weeks of collaboration. We mapped out which parts of the ecosystem needed reinforcement. Some things were urgent: plugin update services, the plugin and theme directories, static assets like emojis and avatars, and the dashboard feeds. We began with mirrors and drop-in replacements, but the plan was always broader than that.

We designed a system that could serve as a complete distribution layer for WordPress. It would work with the core as it exists today, stay compatible while removing the bottlenecks created by centralization, and be governed independently of .org.

FAIR: federated, independent, and live

FAIR is now a technical project under the Linux Foundation. Its technology is governed by a community-led Technical Steering Committee (TSC) and built by contributors from across the WordPress ecosystem. The TSC chose three amazing community leaders as its chairs: Carrie Dils, Mika Epstein, and Ryan McCue. Together, they built a decentralized package management system, federation-ready mirrors, support for commercial plugins, cryptographic signing, and more in a relatively short amount of time.

The goal of FAIR is not to fork WordPress. We’re still using the same core software. We’re not building a separate platform. We are adding a new distribution layer and putting our own governance on top of it. It’s a new path within WordPress, not outside it.

You can still install WordPress from WordPress.org, and that won’t change. But if you want more control over how plugins are delivered, or a system that supports decentralization, FAIR gives you that choice.

This work builds on tools like Composer and package managers from the Linux world, but with a clear focus on usability for real WordPress users. Most people won’t even know how it works under the hood. They’ll just know it works.

Why this matters

When I wrote about WordPress leadership, I meant it. Change in open source doesn’t happen from the outside. It occurs when people show up, do the work, and offer a better option. That’s what we’ve done.

FAIR is not a protest. It’s not a fork. It is a contribution. It reflects our belief that WordPress deserves better infrastructure and more accountable governance, and that we can build that together.

This project results from months of collaboration across companies, countries, and communities. Dozens and dozens of people have already worked on it, and more are joining daily.

You can learn more at fair.pm. There’s plenty of work still ahead, but the foundations are in place, and the path is open.

If you believe in the open web, in WordPress as shared infrastructure, and in a future where the people who contribute get a say in how the platform evolves, join us!

Other people we’ve been collaborating with have blogged as well:

Plus, the press release from the Linux Foundation.

Innovation in WordPress: a look at plugin development

joost@joost.blog (Joost de Valk) — Thu, 05 Jun 2025 00:00:00 GMT

Introduction

Recent observations have highlighted a significant surge in new plugin submissions to the WordPress repository, as noted in this post. We also know that Automattic recently “unpaused” their contributions, leading to some pretty critical articles like this one from Roger Montti.

The increase in plugin submissions got Marieke and me wondering about the relationship between plugin submission, actual plugin availability, and innovation within the WordPress ecosystem. As a result of all the recent changes, WordPress core releases can be somewhat sporadic. Marieke and I wanted to explore whether plugins are now the primary driver of innovation. Marieke took it upon herself to investigate how much plugin development contributes to WordPress’s evolution by going through heaps of changelogs, while I grabbed and analyzed a lot of data from WordPress.org.

This is a different angle on a question I’ve explored before: what the unintended consequences of making SEO accessible have been, and how WordPress’s governance challenges affect what gets built.

Problem statement and research questions

While we see an increase in plugin submissions, it’s unclear if this translates to a proportional increase in plugin installations and overall platform innovation. Many plugins in the repository have minimal installations, suggesting a potential disconnect between plugin development and actual user adoption. With this in mind, we’re asking these research questions:

RQ1: To what extent do new WordPress plugins achieve downloads and active installations, and how do plugins contribute to the innovation of the WordPress platform?
RQ2: Do WordPress plugins compensate for the lack of innovation within WordPress Core?

Preliminary data analysis

When we started digging into the data we gathered from WordPress.org, we found a more nuanced picture than the report suggested. While 2025 apparently shows a notable increase in plugin submissions, historical data indicates peak plugin additions in 2015 and 2016 (see Figure 1). Remember that Figure 1 only accounts for plugins that were approved in those years and are still active in the repository; total plugin submissions for those years were likely higher.

Figure 1## Analysis of all plugins on WordPress.org

To answer research question 1, we analyzed the number of active installs for plugins, grouped by their year of addition to WordPress.org. Figure 2 shows that plugins added in more recent years have far fewer active installs than plugins released years ago. Of course, plugins take some time to grow, so it’s logical that plugins released in 2025 have fewer average installs than those released in, say, 2015. But you might also expect that plugins released now are adding features that people need and would thus grow faster. That doesn’t seem to be happening though.

It appears that it takes years for most plugins to reach a significant user base, if they ever reach it. Plugins released earlier might have had an outsized chance of acquiring users, while new plugins are basically never showing up in search results. Only the ones with external factors (like being recommended by a popular theme or plugin) grow quickly.

Figure 2## Analysis of big plugins

In order to further research our questions, we did an analysis with only the big plugins. In order to make some kind of impact on WordPress as an ecosystem, plugins need to have at least some users. That’s why we looked at all plugins with over 100,000 active installs. What we saw from that analysis is that only five of the big plugins were released in 2024, and just one in 2025. Again, it makes sense that plugins from recent years have fewer installs, but it does suggest that it is hard for new innovations to reach the WordPress userbase through the release of new plugins.

At the end of this post, we’ve some more thoughts about why some new plugins do and some plugins don’t attract a substantial user base.

Analysis of established plugins

It appears that innovation does not come from new plugins. We’ve also established that it does not come from WordPress core. Established plugins maintain a dominant presence in the market, potentially due to longer market tenure, branding efforts, and security hardening.

We’ve looked at the distribution of large plugins and analyzed to what extent plugin usage is linked to the bigger, more established plugins. In our analysis (results in table 1) we researched to what extent the total number of installs (of all plugins across the ecosystem combined) could be accounted for by the larger plugins.

Table 1 reads as follows. In the first column, you’ll read the group of plugins (for example plugins with more than 10 million installs). In the second column, you’ll find the number of plugins in that group, in the third column, you’ll find the number of users from that group and the groups above combined. You can see the cumulative sum of the plugins and the % of installs they have (of the total number of installs throughout all WordPress sites).

Table 1 highlights the concentration of active installations within a few major plugins. This implies that innovation efforts are likely to have the greatest impact within these widely adopted plugins, rather than in the long tail of less installed plugins.

Group**#Cum. #Cum. sum% installs% plugins****Plugins with > 10M installs**4440,000,00011.52%0.01%Plugins with > 5M installs71182,000,00023.61%0.02%Plugins with > 2M installs2132137,000,00039.45%0.05%Plugins with > 1M installs3769174,000,00050.10%0.11%Plugins with > 500k installs66135215,900,00062.17%0.21%Plugins with > 100k installs361496276,300,00079.56%0.78%Plugins with > 10k installs1,9712,467327,120,00094.20%3.86%Plugins with > 1k installs5,6208,087343,199,00098.83%12.65%Other plugins52,60382.30%Table 1

You’re reading that correctly: a group of 4 plugins (less than 0.01% of all plugins) is responsible for about 12 % all plugins installed on WordPress sites worldwide. And, 69 plugins (all having more than a million installs) account for half of the total installs. The usage of plugins in WordPress is highly skewed towards big, well-established plugins.

This leads to the following conclusion: if the big plugins have the most users, then they are the ones that could (and should) drive WordPress innovation. And not the small plugins that hardly anyone uses (yet). So, we need to investigate how these big WordPress plugins are doing in terms of innovation.

Analysis of changelogs

We’ve come to the next step of our analysis, which contains a lot of dull research work. This research employs an analysis of plugin changelogs to assess the extent of innovation within big plugins. Marieke spent hours and hours going through changelogs and classifying them. Examining the release notes for plugins included in this study, changes such as “new features” and “enhancements” are documented over time. It is acknowledged that the terminology used in changelogs varies. Therefore, this methodology is primarily for longitudinal intra-plugin comparison (so with themselves) rather than inter-plugin comparison (with each other).

This is not scientific research (at all). Marieke tried to look at different big plugins while using her background knowledge as a researcher and a WordPress person. We chose plugins that are well-known and that we know well, while trying to get different plugins from different areas. We could have chosen different ones. Marieke wanted to include Learndash too, but couldn’t parse their changelog. Consider this a deep dive into what important brands in the WordPress world do innovation-wise. It’s not complete, but it indicates what’s happening.

Plugin innovation

The Big Automattic Plugins

Looking at WooCommerce, we see they released many new features from 2021 onwards. However, this year they seem to be behind schedule. Perhaps they’re planning something significant, but it does look a bit concerning for 2025.

Also, for Jetpack, 2024 was a low point since 2019, and 2025 isn’t looking much better regarding new or enhanced features.

### The SEO plugins

The three major SEO plugins, Yoast SEO, Rank Math, and All in One SEO (AIOSEO), are substantial projects with large codebases and extensive functionality. They account for over 16 million active installations according to .org’s public numbers (Yoast SEO: 10+ million, Rank Math and AIOSEO: 3+ million each).

Since 2021, Yoast SEO has introduced fewer enhancements, with a notable decline in 2024, and early indications for 2025 suggest a continued downturn. Rank Math exhibits a similar trend, with a marked slowdown in innovation during 2024 and 2025. In contrast, AIOSEO experienced a surge in new features in 2024, but projections for 2025 (based on changelogs so far) also indicate a significant decline.

## Other notable plugins

Marieke then further analyzed the innovation trends of five other notable plugins, revealing a mixed picture. WP Rocket and Elementor have demonstrated steady, consistent innovation, although 2025 shows a slight slowdown, possibly indicating an upcoming major release. Contact Form 7 has exhibited a gradual decline in innovation since 2022. Easy Digital Downloads (EDD) experienced a growth in new features during 2024, yet 2025 appears to lag behind. GiveWP, on the other hand, has maintained a relatively stable pace of innovation. While 2025 shows a dip, this pattern mirrors 2022, and innovation rebounded in 2023.

## What does this mean?

The current state of innovation within the WordPress ecosystem is concerning. Core updates have slowed, and contributions from key players are either leveling or even declining.

While new plugins could theoretically drive innovation, they tend to have limited adoption and install bases. Meanwhile, established plugins, particularly in the SEO space, are not showing signs of renewed innovation; in fact, many are introducing fewer updates than in previous years, and the outlook for 2025 is not promising. This stagnation poses a risk to WordPress’s competitive position in the broader market.

Why don’t new plugins find new audiences?

As said, only 5 plugins in 2024 and 1 plugin in 2025 have crossed the 100,000 installs mark. These plugins are:

ReleasedActive installsPlugin****Built & promoted by20241,000,000+Image OptimizerElementor2024500,000+WooCommerce Legacy REST APIWooCommerce2024200,000+OmnisendOmnisend2024200,000+SureFormsAstra / Brainstorm Force2024100,000+Site MailerElementor2025100,000+SureMailsAstra / Brainstorm ForceAll of these plugins reached their size because of either an enormous marketing effort (in the case of Omnisend) or because they were promoted by other major plugins or themes.

The WooCommerce legacy REST API plugin restores a feature that WooCommerce removed. As of this writing, it hasn’t been tested with the last three major releases of WordPress, which means it was released and then not maintained anymore.

With so many plugins available and many of them not offering excellent quality, how are users supposed to find the truly innovative, useful ones? That’s why we need to do a better job promoting high-quality, innovative plugins in the repository and at events like WordCamps. We should share ideas, show each other how we’re improving WordPress, and celebrate our wins as a community.

Bigger plugins with large user bases and solid business models should also be encouraged to keep innovating, perhaps even by teaming up with new, promising plugins. Hosting providers could play a key role too, by identifying great plugins and helping their customers discover them.

Together, we can ensure WordPress keeps growing and stays an excellent platform for everyone.

The unintended consequences of making SEO “for everyone”

joost@joost.blog (Joost de Valk) — Tue, 22 Apr 2025 00:00:00 GMT

At Yoast, we had one mission: make SEO easier. For a long time, SEO for everyone was Yoast’s tagline, and we meant it. We helped millions of people optimize their content. We made technical SEO more accessible. We gave small businesses, bloggers, and creators a real chance to be found online.

And we were successful. But that success came with side effects. In trying to democratize SEO, we also helped shape habits that led to a web full of “optimized” content, not all of it valuable.

What we got right

A lot of what we did still matters. The readability checks in Yoast SEO were Marieke’s idea, rooted in her belief that to get as big an audience as possible for your ideas, you need to make them more easily understood. We hired a team of language scientists to research and build them. They were grounded in years of academic work on what makes text understandable, including shorter sentences, clear transitions, and active voice. These principles are still incredibly relevant, especially in an AI-powered world.

For instance, we also talked about paragraphs needing to be about one topic (and not too long), with a clear use of headings throughout. SEOs now talk about this and then call it “chunking.” They talk about writing for LLMs as much as for people. But the truth is, this was always about helping readers make sense of content. And it still works.

We also introduced inclusive language checks. That wasn’t about SEO rankings, it was about making the web, and your content, more welcoming. And that still feels like one of the most valuable additions we made.

The deeper foundation

Some of the best things we built were grounded in ideas older than SEO itself. The structure of a good site comes straight from librarianship, from research on findability and classification, and from the craft of organizing knowledge so others can discover it.

Those principles guided how we thought about internal linking, content hierarchies, and sitemaps. They’ve lasted because they weren’t based on trends. They were based on how people find and make sense of information.

What we didn’t see coming

But not everything we did aged well. We turned SEO into a checklist. Our traffic light system, meant to encourage good habits, became a finish line. People wrote to get the green light. They optimized, published, and moved on. Sometimes they were writing with purpose. But sometimes, they were just filling in the blanks.

We encouraged more content, more optimized content. And we made it easy, maybe too easy. That led to a flood of articles that hit the right technical notes, but didn’t always add real value. In our effort to make SEO accessible, we also helped normalize quantity over quality.

We did try to combat the over-optimization; we introduced keyword distribution checks to see if you stayed on topic. We warned against having a keyword density that was too high. But, in hindsight, I don’t think we did so strong enough, for which I’m sorry.

The myths we helped create

Here’s a perfect example:

“Articles should be at least 300 words, because Google wants that.”

Google never said that. I said that.

There was some logic to it. Most algorithms need a little content and context to understand a topic. But it wasn’t deeply researched. It was a practical rule of thumb that got repeated until it became gospel.

And suddenly, people were writing to hit a number. Not because their story or idea needed that much space, but because the plugin suggested it.

The tide has turned

Today, that old playbook no longer works. Publish often and optimize hard? That approach is outdated. At the recent SMX Munich conference, half the talks were about pruning content, removing over-optimized articles, and merging similar articles into fewer, stronger ones.

It’s now clear to everyone there, it seems, that search engines and AI systems favor clarity, structure, and originality over repetition and keyword stuffing. The focus has shifted from more content to better content.

And I think that’s the right direction.

What I’ve learned

The tools we build shape the web. They create incentives, workflows, and habits, some good, some not.

The parts of our strategy that stood the test of time were the ones grounded in real research: readability, findability, and structure. Not the tricks or shortcuts. The fundamentals. The stuff that just makes people build good websites. That’s where I’m focused now.

Where I’m going from here

With Progress Planner, we’re building tools that support thoughtful website SEO strategies. Tools that don’t just help you publish, but help you manage what you’ve already written. Tools that reflect the full life cycle of content: writing, improving, and pruning.

In the previous release, we added nudges that’ll help you set up Yoast SEO better, configuring things that matter, but that we know people often get wrong. We plan to do this for more plugins.

In the upcoming release, we’ll remove features that reward volume for volume’s sake. We’re doubling down on usefulness, clarity, and long-term website performance.

Because SEO for everyone is still a goal worth pursuing. But it has to be the right kind of SEO.

Build websites like it’s 2005 (and win in 2025)

joost@joost.blog (Joost de Valk) — Wed, 26 Mar 2025 00:00:00 GMT

Last week was a whirlwind, first diving deep into AI and WordPress while working with the WP CLI as MCP host team at Cloudfest, then heading off to SMX Munich for non-stop conversations about SEO, AI, and the future of search. Two very different settings, one clear takeaway:

The web has come full circle.

Despite all the advancements in AI, one theme kept surfacing again and again:
These systems still struggle to understand the web.

It’s not just about bad content or spammy sites. Even well-meaning, beautifully designed modern websites are often invisible to AI—and even to traditional search engines—because of how they’re built.

That might sound surprising in 2025, but it really shouldn’t be.

See, for example, this new experiment by Jamie Marsland. He built a headless site with Lovable using WordPress as a backend. My immediate response to his tweet was loading that URL in ChatGPT and it clearly showed it could not see any of the contents of that page, which I, of course, told him too. Let’s dive into why this doesn’t work.

What AIs need (and what they’re not getting)

My friend Jamie Madden pointed me to llmstxt.org. This is a site pushing for standards that help AIs crawl and understand your content better. And the advice there hits hard, because it shows just how bad the situation really is.

If you want your content to show up in any kind of search — be it classic search engines like Google and Bing, or newer AI-driven search tools — it needs to be:

Accessible to bots (so: don’t block it with robots.txt or technical barriers)
Easy to reach (no complex JavaScript rendering or tons of chained requests)
Easy to parse (clear, valid HTML, not div soup)
Easy to understand (simple language, good structure, solid readability)

Sure, Google and Bing have gotten better at dealing with #2 and #3 over the years. But the newer crawlers, the ones feeding AI models and tools? They’re still toddlers learning to walk. They do not execute JavaScript at all.

And now, instead of them getting smarter, we’re being asked to dumb things down for them through new standards like the above mentioned llms.txt.

But here’s a thought: why not just make our sites simpler and better for everyone?

What this really means

This isn’t just about AI or SEO. It’s a reminder of the principles the web was built on: accessibility, clarity, speed, and user-first design. Somewhere along the way, we traded a lot of that for flashy frameworks and brittle complexity.

You can have the most beautifully designed website in your favorite JavaScript framework, but if the HTML that comes out of it is garbage and your content is buried behind five layers of client-side rendering… you’re invisible. To AI. Often even to Google. And thus: to users.

It doesn’t matter how great your content is, how strong your brand is, or how well you target user intent. If machines can’t read your site, people won’t find it.

Funnily enough, if we did this well, accessibility wouldn’t be as much of a challenge and SEO would suddenly be simple.

Back to basics, forward to the future

This moment feels oddly familiar. Like we’ve been here before. It’s the early 2000s again, but instead of optimizing for search engines, we’re optimizing for LLMs too now. And the fundamentals haven’t changed.

So maybe it’s time to build for clarity again: for humans first, and machines second, with systems that honor the web’s foundations instead of working against them.

Because in the end, the best content in the world is useless if it’s unreadable by the systems that help people find it.

Redirect-By HTTP headers

joost@joost.blog (Joost de Valk) — Thu, 13 Mar 2025 00:00:00 GMT

One of the things one runs into when you’re doing large migrations (between domains, or within a domain) is that you run into redirects that go “wrong”. A system somewhere is doing a redirect, and you don’t know which system you need to change that redirect in. A “Redirect-By” header helps you find out which system did the redirect.

Origin story

I ran into this problem while doing the migration of the Guardian from guardian .co.uk to the theguardian .com. Multiple systems were doing redirects, making it hard to find the source. So, I came up with the X-Redirect-By header, this header was sent along with each redirect. This helped me identify which system was doing the redirect.

In 2017 I wrote this blog post on Yoast.com to propose the X-Redirect-By header. We then implemented that, first in Yoast SEO, later on also in WordPress, and other systems like Typo3 followed suit.

Usage of x-redirect-by

Currently, a large portion of sites in the world uses the X-Redirect-By header to indicate which system created a redirect. The Wikipedia page about HTTP headers lists it as a “Common non-standard response field”.

Should we standardize redirect-by?

As I was preparing my presentation for SMX Munich next week, where I’ll be presenting “The missing guide to SEO domain migrations”, I was discussing this with John Mueller. He kindly informed me that x- prefixed headers are actually discouraged. In fact, Google has been trying this for x-robots-tag, see this IETF draft.

Which got me thinking: should we take the next step, and standardize this properly? I’ve been going over the existing standard HTTP headers, and while I think that you could use Via or Server headers for this same purpose, it’s not the same. A proposal for an official Redirect-By header would have the benefit of getting more people to notice it and probably implement it, which would potentially help every SEO and web developer. Lots of them have told me they have spent many hours finding the source of a redirect.

I’d love feedback on this. What do you think? Should we standardize this as redirect-by? Am I missing an obvious, already existing, standard header that could be used for this? Let me know in the comments!

WordPress comments, cookies and caching

joost@joost.blog (Joost de Valk) — Wed, 01 Jan 2025 00:00:00 GMT

This post explains how WordPress uses comment cookies and why that is detrimental to your site’s caching. It then shows you how to fix this.

When I wrote my previous post about WordPress leadership, I had anticipated getting a lot of comments. It turned out there were even more than I expected. This led to some issues with my (admittedly rather aggressive) caching settings on my blog. When I approved a comment it didn’t clear my Cloudflare edge cache of my post, so people couldn’t see them.

A few days after fixing that, I was alerted to the fact that Cloudflare was caching the details of the last commenter on a post. This led me down a rather deep rabbit hole. I was reading code that I’d not seen in quite some time, in the WordPress comment system.

Now, this is all admittedly… A bit of the stuff that makes my kids call me a boomer. Because who does all this commenting on blogs? Right, not my kids. But still, a problem nonetheless.

This is rather lengthy, so here’s a table of contents:

How the WordPress comment system works
The solution: moving comment cookies to JavaScript space
Aside: the SEO impact of the WordPress comment system

How the WordPress comment system works

When you leave a comment on a WordPress site, the process works as follows:

Your comment is submitted;
WordPress sets a cookie for each of your name, email address, and website URL;
Your comment is either held for moderation or immediately approved, based on several criteria. Then WordPress redirects you to a new URL and renders the post you commented on.

The URL it redirects you to can be two things:

If your comment was immediately approved, it’s the URL you started on, with a #comment-<comment-id> anchor so that you’re shown your comment on the page immediately.
If your comment was held for moderation, by default, WordPress sends you to the URL you started on with a unapproved URL parameter + a moderation-hash parameter. It uses these to show your unapproved comment rendered where it would be rendered. It adds a message around it that your comment is held for moderation.

When WordPress renders that page, and every time after that when it renders any blog post on the site for you, your browser sends your cookies to the server, and WordPress fills the data from those cookies into the comment form. So when that view, or a later view, is proxy cached (for instance by Cloudflare), your personal data gets cached along with it.

This is of course highly undesirable on sites that have aggressive caching. On sites like that, you want every page render to be as similar to another one as possible. Even more important you certainly don’t want Personally Identifiable Information in those rendered pages.

The solution: moving comment cookies to JavaScript space

The solution to this is fairly obvious: there’s no reason why this would need to be done server side anymore, as we can do all of it in JavaScript. To do so, we have to take a few steps:

Set the appropriate cookies when you leave a comment (this is going to be in JavaScript);
Prevent WordPress from setting the comment cookies in the first place, as those might get cached;
Prevent WordPress from adding the content of those cookies to the rendered output on the page, as that would get it cached (this will be a few simple PHP functions);
Use JavaScript to read those cookies and fill the form fields.

We could do this in 3 steps if we chose not to be backwards compatible: if we chose different cookie names, we could skip step 3, but I like to keep things backwards compatible.

1. Set cookies when you comment

We’ll start by setting the comment cookies on form submission. This assumes that you’re using the default WordPress comment forms, and thus have the commentform id attribute on your comment forms.

We’ll update this code a bit later to add the functionality for step 4 as well. What this code does is fairly simple: it sets the 3 different comment cookies, with an expiry date of one year, when you submit the comment form. It sets them for / on your domain, so that they’re used on all the comment forms throughout a site.

// Function to set comment cookies.
function setCommentCookies( name, email, url ) {
    const oneYear = 365 * 24 * 60 * 60 * 1000;
    const expiryDate = new Date( Date.now() + oneYear ).toUTCString();

    // Set cookies for comment author data. This (slightly inefficient) cookie format is used for compatibility with core WordPress.
    document.cookie = `comment_author=${encodeURIComponent(name)}; expires=${expiryDate}; path=/`;
    document.cookie = `comment_author_email=${encodeURIComponent(email)}; expires=${expiryDate}; path=/`;
    document.cookie = `comment_author_url=${encodeURIComponent(url)}; expires=${expiryDate}; path=/`;
}

document.addEventListener( 'DOMContentLoaded', function () {
    const nameField  = document.getElementById( 'author' );
    const emailField = document.getElementById( 'email' );
    const urlField   = document.getElementById( 'url' );

    // Set cookies on form submission.
    const commentForm = document.getElementById( 'commentform' );
    if ( commentForm ) {
        commentForm.addEventListener( 'submit', function() {
            if ( nameField && emailField && urlField ) {
                setCommentCookies( nameField.value, emailField.value, urlField.value );
            }
        });
    }
})

2. Preventing WordPress from setting comment cookies

To prevent WordPress from setting comment cookies, we should just unhook the function that sets these:

remove_action( 'set_comment_cookies', 'wp_set_comment_cookies' );

3. Prevent WordPress from reading comment cookies

Next we want to prevent WordPress from adding the content from the comment cookies on the rendered pages. Since there is no way to prevent this functionality from running entirely, we just have to filter it to be empty:

/**
 * To prevent this data from being cached, we don't want to show it server side.
 *
 * @param array $commenter Ignored.
 *
 * @return array With empty values for all three keys.
 */
function ignore_comment_cookies_serverside( $commenter ) {
    return [
        'comment_author'       => '',
        'comment_author_email' => '',
        'comment_author_url'   => '',
    ];
}

add_filter( 'wp_get_current_commenter', 'ignore_comment_cookies_serverside' );

4. Fill the comment form with JavaScript

To fill the comment form with JavaScript, we have to read the values from the cookies:

// Function to read cookies.
function getCommentCookies() {
    const cookies = document.cookie.split( '; ' ).reduce(( acc, cookie ) => {
        const [key, value] = cookie.split( '=' );
        acc[key]           = decodeURIComponent(value);

        return acc;
    }, {} );

    return {
        name:  cookies['comment_author'] || '',
        email: cookies['comment_author_email'] || '',
        url:   cookies['comment_author_url'] || ''
    }
}

And now we update the DOMContentLoaded function we created in step 1 to add this:

document.addEventListener( 'DOMContentLoaded', function () {
    const nameField  = document.getElementById( 'author' );
    const emailField = document.getElementById( 'email' );
    const urlField   = document.getElementById( 'url' );

    // Get the data from the comments.
    const cookies = getCommentCookies();

    // Populate comment form fields with cookie data.
    if (nameField)  nameField.value  = cookies.name;
    if (emailField) emailField.value = cookies.email;
    if (urlField)   urlField.value   = cookies.url;

    // Set cookies on form submission.
    const commentForm = document.getElementById( 'commentform' );
    if ( commentForm ) {
        commentForm.addEventListener( 'submit', function() {
            if ( nameField && emailField && urlField ) {
                setCommentCookies( nameField.value, emailField.value, urlField.value );
            }
        });
    }
})

I have combined all of these in this gist, for easy copy pasting.

Aside: the SEO impact of the WordPress comment system

The fact that WordPress redirects to a URL with an unapproved parameter is incredibly tricky in my eyes; I’m never a fan of URL parameters being added anywhere (as it usually busts the cache, and bloats the URL space), but here it also leads to potential SEO issues if people link to these parameterized URLs. I was able to find quite a few of these with relatively simple Google search:

Since WordPress does not noindex these URLs by default, this is a perfectly sensible way of injecting links into a page and getting (usually nofollow, but still) links from domains that have no idea they’re linking to you, thinking they’ve not approved your comment.

SEO plugins should take note and add noindex to every page that has these URL parameters on it, something I think WordPress core should do too. On my site, I have decided to just not let WordPress redirect to this parameterized URL entirely, something I have included in the gist as well.

Breaking the Status Quo

joost@joost.blog (Joost de Valk) — Fri, 20 Dec 2024 00:00:00 GMT

A vision for a new WordPress era

WordPress is at a crossroads, now even more clearly then when I wrote my previous post on WordPress’s roadmap. I had very much intended to leave this topic alone for a bit until after the holiday break, until, last night, Matt imposed a holiday break on us all.

This holiday break caused issues that needed immediate fixing, but also makes more clear that we don’t have the luxury of waiting.

We, the WordPress community, need to decide if we’re ok being led by a single person who controls everything, and might do things we disagree with, or if we want something else. For a project whose tagline is “Democratizing publishing”, we’ve been very low on exactly that: democracy.

I read the recent open letter by 20 signatories on the Repository who say “we stand with you” and make very clear that they object to the current status quo. I agree with most everything they say, and I want us all to take a step further: let’s get to solutions.

You’ll have heard the proverb “it takes a village to raise a child” and that’s very much how I think about developing CMSs too. You need many voices, many ideas, many backgrounds. You need to embrace diversity. Unfortunately, those with ideas that don’t follow the same direction as our current leader, are being shut down, quite a few even banned, hence the reason that those 19 signatories mentioned above chose anonymity. I get it and I respect it.

Our BDFL is no longer Benevolent, and because of that, speaking up in public is a risk. In an interview with Inc (which to be fair Matt called a “hit piece” himself) Matt said he preferred the term “enlightened leader” over “dictator”. I asked ChatGPT what would be qualities of an enlightened leader, it gave me the following:

An enlightened leader embodies qualities that inspire trust, foster growth, and guide others with wisdom, compassion, and integrity.

I think it’s fair to say that most in the community would disagree with that being an apt description of Matt’s current behavior.

On contributing & influence

While at Yoast, we always contributed “big”. Mostly because we wanted to, but quite often also because we had to: if we didn’t build the APIs to integrate into the things we wanted to integrate with (for example: Gutenberg), we simply couldn’t. As owners of Yoast we’ve spent well over 5% of revenue over the course of my time there on WordPress core development and that tradition seems to mostly continue there today.

In exchange for our contributions, I did get some say in where WordPress went, though never officially, and never when it went in directions that Matt disagreed with. Over time, that influence became less as Matt tightened his grip on the project. I think that tightening was in part a cramp. Wanting to control more what people were working on, because the project wasn’t progressing fast enough in the direction he wanted it to go in.

A lot has happened over the last few months, that I think all comes down to the above. I’ve often considered how the WordPress world “worked” unhealthy. I’ve spoken to many slightly outside of our industry over the past months about what was happening and several people, independent of each other, described WordPress as “a cult” to me. And I understand why.

I think it’s time to let go of the cult and change project leadership. I’ve said it before: we need a “board”. We can’t wait with doing that for the years it will take for Automattic and WP Engine to fight out this lawsuit. As was already reported, Matt said recently in Post Status that “it’s hard to imagine wanting to continue working on WordPress after this”. A few days later, he gave a completely conflicting message in the State of the Word. Yet he never came back on that first statement or clarified that he’d changed his mind. He also didn’t come back to talk to the community he turned his back on.

Last night, he disabled the registration of new accounts on WordPress.org and said this in his post:

I hope to find the time, energy, and money to reopen all of this sometime in the new year.

What if Matt doesn’t reopen registration? Or only does so for people with an @automattic.com email address? Let’s be clear: he could.

I’m still, to this day, very thankful for what Matt has created. I would love to work with him to fix all this. But it’s clear now, that we can no longer have him be our sole leader, although I’d love it if we could get him to be among the leaders.

What should happen now?

In my ideal world, the following things happen:

A WordPress foundation like entity becomes the lead of the project, and gets a board of industry people, from diverse backgrounds.
WordPress.org and all other important community assets that are currently “Matt’s private property” are handed over to that foundation.
The WordPress trademark is given to the public domain or otherwise dealt with in such a way that every company can freely say that they do “WordPress hosting”, “WordPress support” etc. Not just because that’s the right thing to do in my mind, but because doing so means we allow growth of the terms and the concepts.
People and companies can become sponsors of this WordPress foundation like entity, and we give some perks to those sponsors (for instance, listing them on a hosting page) and we disclose all of that transparently.
We create several small teams responsible for Architecture, Product, Events, etc. and create a proper governance structure around this.

There’s lots more to figure out, but these are steps that must be taken fast. On the code side we need to take steps too, and we can most quickly act on one that has become very clear that we need immediately:

Federated And Independent Repositories

We need to supplement WordPress.org updates with other sources, so that what happened to Advanced Custom Fields, can’t happen again. Lots of hosts are currently experimenting with or already putting in place mirrors of WordPress.org. This creates issues: download and active install statistics are no longer reliable, for instance.

Just having mirrors of WordPress.org also doesn’t really solve the problem of a single party controlling our single update server. For that, we need to make sure that those mirrors federate with each other, and share each others data and, as Karim suggested, allow for independent plugins and themes to be hosted there, outside of the wordpress.org repository. I call this: Federated and Independent Repositories, in short: FAIR.

I’m already talking to several hosts about this, and would welcome anyone who wants to join these conversations, so we’re not duplicating work.

Matt might not agree to my first five points above. However: we can still work on the Federated and Independent Repositories without his permission, because frankly, we don’t need it.

We take back the commons.

What is my role in this?

I’ve already been working and talking to lots of people over the last few months trying to find ways out of this. I want to hold this community together. This resulted in several conversations, starting with my conversations with Matt and how to reconcile, to several conversations I’ve initiated with key community people & hosting leaders. This led to a deeper conversation with Karim Marucchi, CEO of Crowd Favorite, one of the WordPress world’s first enterprise-grade agencies, about breaking away from the status quo and focusing on the future. I feel this gives me a base for moving the conversation forward with people that I can fall back on and who can help to transition us to what’s next.

Taking back the commons means that we try to hear every voice, be considerate of all the different use cases of that commons and bring us all forward. We may not all agree along the way, but we’ll talk about that openly, without fear. We may make mistakes, and then we’ll set them right, and if needed, apologize.

I’m here, and willing to lead through this transition. I do have the time, the energy and the money needed to fund myself doing it. I’ve worked in this industry and this community for close to 20 years and it’s very dear to me. Thanks in large part to the WordPress project, I have the privileged position to be able to drop and/or delegate some of the stuff I’m working on and start working on this.

Let me be clear though: we should not replace one BDFL with another. This is a moment of transition. I’m also very willing to work with other leadership if it turns out the community wants someone else.

If you haven’t already, go read Karim’s post too and let me know all your thoughts. In the comments here, on Bluesky, X, through DMs somewhere or through my contact form here.

What happens now?

Right now, in the next few weeks: nothing. We do, truly, all deserve this holiday break (for those it applies to). I’m going to celebrate the holidays with my wife, children and wider family. I hope everyone can reflect about this throughout the holiday season (or you block it out entirely, and think about it later, fine too!).

We (being Karim and myself) will start 2025 by getting into a (probably virtual) room half way through January with lots of the leaders of this community & industry and decide where we go from here. So, enjoy, rest, and get ready!

WordPress, and what should be on its roadmap

joost@joost.blog (Joost de Valk) — Thu, 12 Dec 2024 00:00:00 GMT

Update: Since writing this, I proposed FAIR as a path forward for WordPress governance, and later wrote about why we stopped pursuing it.

I was reading Hendrik Luehrsen’s excellent post “WordPress isn’t WordPress anymore“, and I decided I had to write more about this. I recently spoke at WordCamp NL about “The missing features of WordPress”, and these two things “touch”, in an important way. I love WordPress. I love WordPress plugins. I don’t love some of the recent developments in WordPress and that’s not just talking about the recent Automattic – WP Engine drama.

When I started writing this, there had been no decision on the injunction yet. Now there is, and as a community we have to figure out how we deal with all this. I don’t think we can separate that entirely from looking at what we all want from (and for) WordPress. Good discussions about where WordPress should be headed have been lacking from the community for a while.

So: let me share my thoughts, first by looking at WordPress’s market share and the trends in the wider market, then by discussing what WordPress is, or should be.

Looking at WordPress’s market share

If we look at WordPress’s market share, we have to admit that it’s stable at best. WordPress, which has been growing for the longest time since it was created, is now stagnant. The current market share (as of November 2024) is slightly below what we started the year at:

This means there’s also hardly any growth since early 2023, which should be (and probably is to everyone who’s aware) concerning to everyone in the WordPress industry.

To make things worse, all other open source, self-hosted technologies are losing:

If you’re now thinking “but then other projects must be winning”, you’re 100% right: SaaS tools are winning. Just look at the numbers:

I’ll give my sharper analysis: if WordPress hadn’t had Elementor, it would have shrunk (something I’ve been saying for a while). This is definitely also true, though with less big of an impact, for WooCommerce, the other driver for WordPress growth. Which shows that it’s not that people don’t want Open Source. Let me show this with a graph too:

So, it’s not that people dislike or don’t want Open Source. I honestly think people don’t care that much about this at all. I think we have other problems. Let’s dive deeper.

Why is this happening?

The question we have to ask ourselves of course is: why is this happening? What are people and companies looking for that WordPress is not providing? I think there are several important factors to discuss around this. Many of the arguments that people use against WordPress, we are very often quick to dismiss: WordPress is old, not modern. WordPress is insecure. WordPress is slow. There are certainly more pain points, but let’s try to cover these for now.

WordPress is slow & technically stagnant

WordPress is often ridiculed by developers in the wider web ecosystem. It’s considered backward, slow and bloated. And the truth is harsh but simple: this is true. We have failed to modernize large parts of our codebase over the last decade, especially on the PHP side. Many projects and potential improvements have been discussed, but we focused instead on building features (like the duotone filters Hendrik mentioned). Fantastic work has been done to keep WordPress working with modern versions of PHP, but we’ve failed to truly deal with our technical debt and with that also failed to make WordPress that much faster.

On the JavaScript side, we’re doing the opposite: we’re churning through code harder than we should. The speed of development is outpacing what even some of the best developers can follow if they’re not on the project full time. We bring new JavaScript APIs very often, developing our own instead of integrating existing projects.

WordPress is also at the absolute forefront of some of the web’s development though. The Performance team includes some of the world’s very best frontend performance engineers, who jointly made WordPress much, much faster over the last few years. The problem is: our competitors are doing the same. Shopify improved more than we did. Part of the problem here is that we’re simply not merging the fantastic work the Performance team is doing into WordPress core fast enough.

WordPress is insecure

WordPress is very often dubbed insecure by its competitors. The standard response to this is “WordPress core has hardly had any security issues the last few years”. We have to be honest here: WordPress is often considered insecure because it gets hacked a lot. It doesn’t get hacked through WordPress core, it gets hacked through plugins most of the time. But we don’t have the luxury of saying that those plugins are not WordPress. They very much are. Plugins are, as Hendrik so poignantly discussed in his post too, the reason for WordPress’s popularity, this easy extensibility is also its challenge. You can’t look away from their shortcomings yet benefit from what they do.

Recently PatchStack (disclosure: we’re an investor as Emilia Capital and I’m on their board) held a special event in their Bug Bounty program for Cyber Security Month and they closed 1,000 plugins on WordPress.org as a result. Now that is a herculean effort, but it also leaves everyone wondering: what would happen if we looked at all plugins a bit more deeply?

It also makes you wonder: why can’t we see in our WordPress admins that those plugins are closed on WordPress.org? Why is WordPress, the software, not telling us that plugins we’re using that we installed from WordPress.org, have been closed for security reasons? The ticket requesting this is now officially over a decade old.

A step further would even be to say: this set of plugins hasn’t been updated in the last year. The plugin directory talks about 59,000 plugins. The XML sitemap for the plugin directory tells me that only 14,648 of them have been updated in the last year or so.

WordPress is too hard to use

The last, most painful argument that I want to cover is that WordPress is too hard to use. Currently, WordPress forces users to learn several different admin UIs, with different design languages. WordPress has for years tried to get people to use the Block Editor and more recently the Full Site Editor, and the harsh truth is that the editor that has grown the most in that time is a third: Elementor. See for yourself:

Note that the Block Editor was released December 6th, 2018, now 6 years ago, but wasn’t tracked in this data until earlier this year, and that the drop in Elementor’s growth is the result of a measurement change.

One of the best additions to WordPress’s marketing recently has been the flurry of activity on the WordPress Youtube channel mostly by Jamie Marsland. I have to admit to have been utterly dismayed when I saw Matt Mullenweg battle Jessica Lyschik recently, because it showed so clearly what many people have been saying for a while now: the block editor is too hard to use. Matt resorted to use AI to help him create patterns but that didn’t help him all that much. If Matt, who leads this project and should be pretty well versed in it, can’t figure this stuff out faster, shouldn’t we take that as a sign?

So what to do?

Instead of dismissing this feedback, we should take it head on. If the WordPress performance team has shown us one thing, it’s that we can improve. Now, we must. But to do that, we must solve our existential crisis first:

What is WordPress?

Hendrik in his post, underneath a heading asking “Who is WordPress for” says this:

Historically, WordPress didn’t try to be everything for everyone. Its lean core and robust ecosystem allowed it to meet a wide range of needs without dictating how it should be used. Developers could tailor it to clients. Hobbyists could start small and scale up. Agencies could rely on its flexibility to create bespoke solutions. WordPress provided the tools, but the user always defined the experience.

The “user” over the years have ranged from individuals, small & large businesses up to large enterprises. The power of WordPress is exactly that: it is what you want (and dare to dream) it to be. The problem is that the last few years, new features that went into core didn’t always seem to fit that “lean core” idea.

Of course there are features the enterprise would want, like personalization, logging, A/B testing, etc. There are features that not just enterprise but more sites would like to see, like multilingual support, custom fields, better admin UIs and many, many things. Most of those things do not need to be in core. Plugins have always been a source of innovation and they should continue to be just that.

WordPress core needs to provide the underlying APIs that allow for extension and force some regularity in how we do these things. I was a big proponent of the REST API when it was developed. In the same way, I’m now a big proponent of merging the fantastic work Felix has done on the AI services plugin into core, or at least make that a canonical plugin. I think multilingual support should definitely be built into core, at the very least the underlying APIs.

We also need a new admin UI and a design language that we force on plugins (and themes) a bit harder. Plugins simply should not have (to develop) their own admin UI, a point I’ve been stressing for years. More broadly speaking, we need to work on UX everywhere and make more logical decisions. For example: “Edit site” does not need to be a button in my admin bar. Nobody edits their site’s templates every day, and nor should they.

I’d love to have more of these thoughtful conversations, where we all openly and freely discuss where WordPress should go. So: let’s see your blog posts, outlining your wishes for the project!

Update Dec 20th: I followed up this post with another one with a vision for a new WordPress era.

Can I safely use AVIF or WebP share images?

joost@joost.blog (Joost de Valk) — Wed, 04 Dec 2024 00:00:00 GMT

AVIF and WebP are efficient image storage formats. They are smaller than their predecessors, PNG, JPG, and GIF. This leads to smaller images, which means faster page loads, which is what we all want. So, you should use them everywhere. Ideally, we’d also be able to use these image formats for our OpenGraph image tags (og:image) and Twitter image tags (twitter:image).

In fact, some platforms, like WhatsApp, limit the size of a share image, so smaller would be much better.

Update December 2024
LinkedIn now supports WebP, making all the major platforms support it. I’ve also added Bluesky, it too supports WebP.

TL;DR

You should use both AVIF and WebP on your website. They’re smaller, they’re better. All browsers support them. As of December 2024, you can use WebP share images, but you can not use AVIF on social platforms yet. We still have to get quite a few more platforms to support AVIF before it’s safe to use them as share images. However, changed from June 2024, you can now safely use WebP everywhere.

Test results
Methodology: how I tested AVIF and WebP
How to use AVIF and WebP on your WordPress site
- Modern images in the <head>

Test results

I’ve run a quick test across the platforms below, and as you can see, we still have to get multiple platforms to support AVIF:

Platform	AVIF	WebP
Bluesky	❌️	✅️
Discord	❌️	✅️
Facebook	✅️	✅️
iMessage	❌️	✅️
Pinterest	✅️	✅️
LinkedIn	❌️	✅️
Mastodon	❌️	✅️
Slack	❌️	✅️
Threads	✅️	✅️
Twitter / X	❌️	✅️
WhatsApp	✅️	✅️

Methodology: how I tested AVIF and WebP

I’ve used these two test URLs:

</research/images/avif-test.html>
</research/images/webp-test.html>

With these URLs, I checked whether the platform would load the AVIF / WebP og:image, or show the fallback.

Below are some examples of what that looks like:

How to use AVIF and WebP on your WordPress site

To use these modern image formats on your site you can install the Modern Image Formats plugin. This plugin has been built and is maintained by the WordPress Performance team, which includes some of the very best performance engineers on the web.

When you install this plugin, go into its settings, and set them like this if you want to make sure your images are always recognized by social platforms:

Modern images in the <head>

Alternatively, if you set this to AVIF, you have to make sure that the output in the <head> section of your site always uses the “classic” format, usually PNG or JPG. The Modern Image Formats plugin actually already tries to do this by default, but it unfortunately fails to do that when using Yoast SEO.

The following tiny plugin makes sure that your og:image and twitter:image tags do output GIF, PNG or JPG files:

<?php
/*
Plugin Name: Fix modern image formats for social
Plugin URI: /use-avif-webp-share-images/
Description: Replaces og:image and twitter:image URLs in Yoast SEO output ending in 
             .webp or .avif with the original in the page output, so they work.
Author: Joost de Valk
Version: 1.0
*/

function joost_filter_modern_social_image_url( $image_url ) {
    return preg_replace( '/^(.*)-(gif|jpeg|jpg|png)\.(avif|webp)$/i', 
                         '$1.$2', $image_url );
}
add_filter( 'wpseo_twitter_image', 'joost_filter_modern_social_image_url' );
add_filter( 'wpseo_opengraph_image', 'joost_filter_modern_social_image_url' );

This small plugin also replaces WebP files, just to make sure it works everywhere.

The People Behind the Platform: How WordPress’s Community Drives its Success

joost@joost.blog (Joost de Valk) — Thu, 31 Oct 2024 00:00:00 GMT

By Marieke van de Rakt and Joost de Valk

WordPress marketing is a collaborative effort. WPBeginner, Yoast, BobWP, Siteground, Post Status, and the Repository are just a few of the many, many brands and individuals advocating for WordPress.

We’re incredibly proud of the WordPress community. Amid the recent drama, we feel the community’s role and importance are severely overlooked. Our community comprises fantastic people who have also excelled in marketing WordPress, helping it grow into the world’s largest CMS. In this blog post, we’ll discuss our view on the impact of the WordPress community on the growth of WordPress.

Make the Pie Bigger

During his traditional Q&A session at WordCamp Europe (Paris, Vienna, one of these), Matt Mullenweg spoke about WordPress’s growth. He encouraged us to “make the pie bigger,” something he’s said since at least 2014, emphasizing that as WordPress grew, everyone in the community would benefit from our collective efforts. That phrase, “making the pie bigger,” became our guiding principle.

At Yoast, we fully embraced this idea. We shared updates about WordPress on our blog, newsletter, and social channels. At every SEO conference we attended, going back to 2006, we advocated for WordPress, promoting its benefits to a broader audience. We also contributed to WordPress core, developed free plugins, translated strings, and helped organize WordPress events. All of these actions supported the growth of WordPress.

Automattic and Matt Mullenweg have undeniably had a tremendous impact on WordPress software. Everyone — hosts, plugin developers, agencies — benefitted from their contributions to WordPress. In return, Automattic also thrived thanks to the marketing power of the WordPress community. WordPress (and, with it, WordPress.com) wouldn’t be as widespread without the passionate promotion by the countless WordPress brands and community members. The strength of the community’s marketing has been pivotal in our collective success.

How Marketing Works

To truly appreciate the role of the WordPress community in driving the growth of the world’s largest CMS, it’s essential to understand the science of marketing that backs it up.

Studies show that marketing messages are much more effective when they come from independent sources or trusted opinion leaders rather than directly from the brand (e.g. Hou, 2022, Momtaz et al., 2011). This indirect approach, called social proof, makes audiences more likely to trust the message as they see it endorsed by others (Cialdini et al, 1999). When a recommendation comes from a figure they respect, like an expert or a community leader, it carries even more weight, increasing credibility and influence within the audience’s social circles (Wojdynski & Evans, 2016).

Direct brand messaging often has a limited impact on changing consumer behavior immediately. Instead, it shapes opinions over time through key figures in social networks (Katz & Lazarsfeld, 1955). According to the “two-step flow” model, messages reach the so-called opinion leaders first, who then interpret and share this information with their communities. People are more likely to trust and act on the recommendations of these figures, whose endorsements feel more personal and relevant to their needs (Berger, 2014).

Marketing messages hardly ever reach people directly from the main brand. To successfully grow a brand, one needs to convince the opinion leaders. Let’s look at how the WordPress brand grew, shall we?

Bloggers, Believers, and Brand Builders: The Early Advocates of WordPress

WordPress grew mainly because a vibrant community of early bloggers acted as influential voices, what researchers call opinion leaders or influencers (e.g., Kavanaugh, 2006; Yefanow & Tomin, 2020). Initially, WordPress was a platform for bloggers: a hub for influential voices to connect and create. Many users shared their enthusiasm for the software, writing posts and spreading the word. Notable figures like Matt Mullenweg himself, Lorelle VanFossen, Liz Strauss, Chris Lema, and Syed Balkhi were among these early advocates. Their blogs reached thousands and inspired others to start their own WordPress blogs, gradually creating a vast network of niche blogs, each speaking to a unique audience but all promoting WordPress.

This early community advantage set WordPress apart from other CMS platforms by fostering a loyal, decentralized network of opinion leaders. Each community leader shared WordPress’s benefits in their own way: Joost and Jono Alderson spoke at numerous SEO conferences, introducing WordPress to new audiences, while agency owners from companies like HumanMade and 10up spread the word to clients. Hosting companies like WP Engine and Pagely organized events like Decode and Pressnomics, which were dedicated to educating and inspiring their communities about (the business of) WordPress.

The strength of WordPress’s growth strategy came from its network of bloggers, speakers, and opinion leaders, whose enthusiasm made promoting WordPress feel almost effortless. Their collective support was WordPress’s marketing machine, showcasing how “many hands make light work.”

Influencer Marketing: How WordPress Built a Movement

Today, influencer marketing is enormous. We often think of celebrities endorsing fashion brands or luxury goods getting commissions in return. However, WordPress had influencers long before it became a trend. Bloggers, speakers, and community leaders championed WordPress without needing pay—they were passionate about the platform. This genuine belief in WordPress as the best CMS made their message far more powerful and persuasive.

WordPress YouTubers: Reaching a New Generation

Today’s WordPress influencers reach new audiences, especially on YouTube, appealing to a younger, often less tech-focused crowd. Creators like Ferdy Korpershoek, Darrel Wilson, and WPcrafter have hundreds of thousands of subscribers, gaining viewership far beyond the official WordPress channel.

These YouTubers are compensated for their content through advertising and affiliate partnerships, yet WordPress doesn’t pay them. Instead, they’re supported by companies and brands in the WordPress ecosystem, such as Elementor, GoDaddy, Yoast, and Awesome Motive. This diverse sponsorship shows how WordPress has inspired a community-driven network of old and new advocates essential to its growth.

WP Community and Automattic – symbiotic relationship

Many WordPress companies would never have grown without Automattic’s unrelenting efforts to improve WordPress, the software. However, Automattic, WordPress.com, Woocommerce, and Jetpack would not have grown so much without the unrelenting marketing efforts of the WordPress community.

The growth of WordPress is a testament to what a passionate community can achieve together. Automattic’s ongoing software innovations and the community’s commitment to spreading the word have created a symbiotic ecosystem that benefits all. WordPress’s success story shows how community and collaboration can make an impact far beyond what any organization could accomplish alone.

In Conclusion: A Community-Driven Legacy

The WordPress journey from a simple blogging tool to the world’s leading CMS has been made possible by the passion and dedication of its community. From early bloggers to today’s YouTube influencers, countless voices have spread the WordPress mission. This collaboration — where Automattic’s innovations and the community’s marketing meet —demonstrates that true growth comes from empowering others. Together, we’ve created something much greater than any one brand or platform alone could achieve.

Let’s hope that WordPress remains a tool by the people, for the people.

For more on how we approached WordPress marketing, see the posts on leading marketing for WordPress and the first steps we took.

Transparency, Contribution, and the Future of WordPress

joost@joost.blog (Joost de Valk) — Thu, 26 Sep 2024 00:00:00 GMT

Update: This post was the start of a longer series. I followed it with posts on WordPress leadership, what should be on the roadmap, and eventually a path forward through FAIR.

When we sold Yoast to Newfold in 2021, I quickly learned we had been incredibly naive. While at Yoast, I hadn’t realized how many deals were made between big companies to promote their products. This was my first realization that not all companies thought and worked like we had been doing. I did learn a lot from the way Newfold Digital runs its businesses. Positive things!

But I remember the way I felt back in 2021 so well. I was so disappointed in the community, these large WP businesses, and even the world. And I don’t want to presume that everyone is as naive as I was, but I think there’s a big gap between the community and the ‘big WordPress companies’. It is very hard for community members to understand why these companies are making so much money and are still fighting. So why is that? And is there a solution?

Inequality in information

There’s nothing wrong with making business deals to promote each other’s stuff. However, most members of our community (including me up until 2021) are not aware of those deals. There’s inequality in information, which makes it hard for community members to understand the decisions that big companies make — decisions concerning contributing to WordPress, for example. If we were more transparent about how money flows, it would be a little bit easier to understand.

Different objectives

Corporations that make hundreds of millions of dollars have different objectives than individuals and small businesses working in the community. These corporations have shareholders and other stakeholders, and those stakeholders want to make money. Again, there’s nothing wrong with that.

However, many people in the WordPress community are driven by completely different objectives. They tend to be excited about open source, they want to make a better product or they want to organize an event for our amazing community. When we became part of Newfold Digital, I noticed the difference in objectives. All of a sudden, it became so very clear to us. I know now that I am just driven by different objectives: both Marieke and I don’t get excited about only making more money.

Inequality in compensation

Differences in objectives and inequality in information lead to very big differences in the amount of money that people make from WordPress. On the one hand, there are people who are in the community, volunteering their help and organizing events. In many cases, these people aren’t even compensated for their hours. On the other hand, there are people who are working on the boards of very large companies with very large salaries.

The people who make the most money in WordPress are not the people who contribute the most (Matt / Automattic really is one of the exceptions here, as I think we are). And this is a problem. It’s a moral problem. It’s just not equitable.

I agree with Matt about his opinion that a big hosting company such as WPEngine should contribute more. It is the right thing to do. It’s fair. It will make the WordPress community more egalitarian. Otherwise, it will lead to resentment. I’ve experienced that too.

At Yoast, we contributed many many hours. During the first years of the Gutenberg project, our revenue growth dropped because we couldn’t develop new features for our product due to contributing so much to core. But we still made a lot of money. So, I didn’t really have a reason to complain, did I?

At the same time, we were annoyed that other companies – much bigger than we were at the time – did not participate as much. We felt we were helping them get bigger while our growth stayed behind. I truly do understand where Matt is coming from.

Solution: transparency & governance

So I agree with Matt that big WordPress companies should have to contribute — preferably a lot, just like Automattic and some others do. But I would like to do that more openly: let everybody see how the money flows.

Currently the way it works is that the money for trademark deals flows to Automattic, but we don’t know how much of the contributions Automattic does are paid for by Newfold, whom we now all know are paying for the use of the trademark. Maybe the money should go directly into the foundation? If not, I think we should at least see how many of the hours contributed by Automattic are actually contributed by Newfold.

No taxation without representation

If we require everyone to contribute, we can call that taxation. It’s not as bad a word to me as to some other people, I think, but I do know that with taxation should also come representation.

In my opinion, we all should get a say in how we spend those contributions. I understand that core contributors are very important, but so are the organizers of our (flagship) events, the leadership of hosting companies, etc. We need to find a way to have a group of people who represent the community and the contributing corporations.

Just like in a democracy. Because, after all, isn’t WordPress all about democratizing?

Now I don’t mean to say that Matt should no longer be project leader. I just think that we should more transparently discuss with a “board” of some sorts, about the roadmap and the future of WordPress as many people and companies depend on it. I think this could actually help Matt, as I do understand that it’s very lonely at the top.

With such a group, we could also discuss how to better highlight companies that are contributing and how to encourage others to do so.

Parting thoughts

This is quite a long post. This week has been rough. But as I blog this, I also realize that I’m very thankful for what WordPress has brought me, and all the friendships I’ve been able to create because of it. I want to thank everyone whom I’ve been discussing this with the last few days for all their insights and for taking the time to talk to me and help me find my bearings on this rough sea. I also want to thank my wife Marieke, who is just the very best, and helped me put these words together.

CMS market share June 2024 commentary

joost@joost.blog (Joost de Valk) — Tue, 02 Jul 2024 00:00:00 GMT

This commentary focuses on the data found in my monthly CMS market share report, which is generated automatically every month. This is the first time I’m writing the commentary separately from the data, so I’m still figuring things out, but as I was building this, I had so many interesting tidbits that I wanted to share immediately, but I also really had to finish the code for the report. So, here it is, all these things I noticed:

Squarespace’s advertising works?
Tistory: a new kid on the block?
Woo vs. Shopify vs. more?
Elementor keeps on growing
Adobe is not so lucky

Squarespace’s advertising works?

Anecdotally, I heard about Squarespace advertising more, so their growth could be due to their advertising. But it could also be something else. Their growth chart shows that whatever it is, it’s working. Note that these effects can rarely be immediate; a site needs to have reasonable traffic to start showing up in this dataset, which normally doesn’t happen a month after launch:

Tistory: a new kid on the block?

I had personally never heard of Tistory before, but it turns out it’s been around for about 18 years, it just only showed up in our measurements. This is a good example of the gaps in all of the data sources. I hope we’ll be able to make this report more comprehensive over time and not run into gaps like these any more.

Woo vs. Shopify vs. more?

For a while, it seemed like the only two real eCommerce contenders were WooCommerce and Shopify. But looking at this data, Wix’s eCommerce offering is actually also growing very rapidly and Squarespace’s Commerce solution is also not to be scoffed at, both only entered the market in 2021:

The individual year-on-year growth rates aren’t too obvious from this chart. But if you calculate them for June 2024 vs. June 2023, they were showing quite a stark difference:

WooCommerce5.6%Shopify15.5%Squarespace Commerce11.7%Wix eCommerce26.9%## Elementor keeps on growing

One company in the WordPress space feels like it has no limits to it in terms of their growth: Elementor. Their growth chart over the last years can be described as “up and to the right”, with their most recent YoY growth rate at 23.3%:

However, if you look at the number of installs for Yoast and Elementor, it looks like Elementor has surpassed Yoast judging by some of the numbers outed by both companies. However, Yoast is still much bigger in terms of market share judging by this dataset (note we only have data for Yoast SEO as of January 2020):

Adobe is not so lucky

Adobe acquired Magento about 6 years ago, and it seems it’s not been able to make it “work”. Similarly, Adobe Experience Manager, which in my personal opinion is an overpriced solution that lacks some of the most basic functionality to manage a site (particularly on the SEO front), was on a path downwards up until May, although it seems to have recovered a tiny bit in June:

That’s it for this month; I’d love to hear your thoughts and opinions on the data, the new format and questions you may have!

CMS Market share measured “right”

joost@joost.blog (Joost de Valk) — Tue, 02 Jul 2024 00:00:00 GMT

Today, I’m introducing a new CMS market share report on this site based on the HTTP Archive’s dataset. This report will be updated – automatically – every month. It doesn’t just contain CMSes; it also contains the most popular eCommerce platforms, WordPress page builders, and SEO plugins. Over time, we might expand this to more; suggestions are most certainly welcome.

I had been slacking for more than a year in publishing a new version of the CMS market share statistics, partly because of time but mostly because I was unhappy with the data I was dealing with. In the last few weeks, I sat down and thought about it and discussed it with Jono Alderson (whom I owe massive thanks for his help as he was always the one helping me with my previous CMS market share reports). The conclusion was that we need to build this better. So we did.

The report now also contains a list of all the technologies we track, allowing you to click on pages for each.

This replaces the analysis in my earlier posts on WordPress's shrinking market share and Elementor as a growth driver, both of which relied on W3Techs data.

Separating data from commentary

In my previous reports, I added my commentary throughout the report. However, as this report will now be updated automatically every month, that’s no longer possible. So, instead, I will post more regularly on this blog in this category.

The first post is straight after this one: CMS market share June 2024 commentary.

On measuring market share

Measuring market share on the web is incredibly difficult: you need to define what “the market” is, and even the definition of a “CMS” is being discussed in some places. This report now assumes that the Chrome UX dataset, which is the basis for the HTTP Archive’s dataset, makes the right decisions to get a statistically reasonable “view” of the web. This does mean, however, that very small sites (in terms of traffic) are not included in this dataset, which in itself leads to some bias.

WordPress

The biggest visible change in all of this is that WordPress has a lower market share than the numbers we’ve previously shared, which were based on W3Techs dataset. The difference is due to many things, including choosing whether to count subdomains (W3Techs counts all subdomains of a domain as 1 site).

As always, with data like this, you should watch the trend and its direction.

This is the data we have for WordPress in this dataset:

Measurement fixes

While rebuilding this report, I made several pull requests on the HTTP Archive’s clone of the Wappalyzer project to improve measurements and fix categorization.

For instance, the SEO plugins report shows that All in One SEO is incorrectly measured. It’s been consistently going down, and I know from other sources (including talking to Syed) that while it did go down for a while, it’s been going up again. It turns out that its measurement was broken, so I made a pull request to fix that.

Enjoy the new report, my first analysis post and let me know in the comments or on Twitter or elsewhere what you think!

Introducing a new plugin: Progress Planner

joost@joost.blog (Joost de Valk) — Fri, 07 Jun 2024 00:00:00 GMT

Ever since Marieke and I left Yoast, we’ve considered building a new WordPress product. We love working on something that people can use—something that will improve their website or make working on their website easier. About a year ago, we came up with something, and we’ve been playing with that idea ever since. We’ve been creating and developing in the past few months, and today, the first version is done! Let me tell you everything about Progress Planner!

Planning and organizing

Both Marieke and I aren’t particularly good at planning and organizing. Marieke noticed that when leaving Yoast. Over the years, we have built-in strategies and tactics for getting our own work organized. Managing a website consists of all these tasks that people easily forget or postpone. Could we build something that would make working on your website easier and more fun? Something like Duolingo or Fitbit for your website. That’ll keep you motivated to update and add stuff.

So, that’s what we did. We built a plugin that motivates you to keep up the work on your website. We give you all kinds of stats on your activity, and you can unlock badges and keep streaks. Your activity score will go up if you update plugins or add content. Every Monday, we’ll email you with your progress and motivate you to keep going.

A screenshot of Progress Planner’s first badges# Pro version: helping people even more

There will be a Pro version at some point later this year. We’re already developing features for that. Our idea is that in addition to motivating people to keep going with their websites, we should also help them with the work. We’re building several mini-courses that will be accessible from the WordPress backend, right in the edit screen. In those courses, we’ll help people how to draft or improve certain pages. How do you write a proper ‘about’ page? What does a good FAQ page contain?

In addition to these courses, we’ll give people direction and help them decide what task to start next. We’ll also help them set goals and come up with proper to-do lists. The whole website maintenance thing can be rather overwhelming. We want to direct people to what tasks to take on next.

A screenshot of Progress Planner’s Website activity score# Small features – significant improvements

We’ve made a nice dashboard in the WordPress backend that allows for a small to-do list and will tell you when to update your plugins. Marieke updated the plugins on her site for the first time (I usually take care of it because she keeps forgetting). You’ll collect points for updating, which motivated her to do it herself. This is going to save me a lot of work!

We’ve also added a to-do widget to the WordPress dashboards, allowing you to write down the tasks you want to work on and see them every time you log into your site. This is a small feature, but it will certainly help me to remind myself of the things I need to get done on my site.

A screenshot of Progress Planner’s to-do list dashboard widget# Try out our beta!

We’ve just released the very first version of Progress Planner on WordPress.org. We’re very excited because it looks amazing and already helps us keep working on our websites. We even get a little competitive! Please try out the beta version of Progress Planner and let us know how to improve it further.

A right-click / contextmenu logo popover

joost@joost.blog (Joost de Valk) — Mon, 27 May 2024 00:00:00 GMT

I always like it when I right-click a logo on a website because I need to get the logo, and the site’s designer has thought about this. If you right-click on the logo on our Emilia Capital site, you’ll get a popup that sends you to our logo page. As of today, that’s not a popup, but technically a popover, using the new popover API in HTML.

An example of what we’re going to build in this post.## Step 1: the HTML

The HTML for this is extremely simple, and the magic is in line 1:

<div popover="manual" id="logo_popover">
	<h3>Looking for our logo?</h3>
	<p>You're in the right place.<br/>Check out <a href="/logo/">our logos & style page</a>.</p>
	<button popovertarget="logo_popover" popovertargetaction="hide">Close</button>
</div>

This is a popover; if you haven’t heard about it yet, popover is the new, browser-native way of doing modals, which hugely simplifies building modals. Just how much it simplifies it? Well, if I wanted to open the above popover on click, all I’d need to do is add this somewhere:

<button popovertarget="logo_popover" popovertargetaction="show">
  Open popover
</button>

In this case, we’re not going to open it with a button in the HTML, but rather on *right-*clicking the logo, so we need the tiniest bit of JavaScript.

Step 2: the JavaScript

With JavaScript, we’re going to catch the contextmenu event of the logo. This means we’re going to grab right clicks on that element. The JavaScript is actually super simple, too. We find the logo with document.querySelector. We attach an event listener to the contextmenu event of that logo. Then, we prevent the default (which is showing the context menu). Lastly, we find the above popover by its id attribute and make it show up.

<script>
document.querySelector(".is-logo-image").addEventListener('contextmenu', function(event) {
	event.preventDefault();
	document.querySelector("#logo_popover").showPopover();
	}, false);
</script>

If you do not set the popover above to close manually, so instead of popover="manual, you just put popover, you may find that the popover closes immediately.

Step 3: the CSS

There’s a tiny bit of styling needed to determine the color of the backdrop for your popover. Simply do this for a black backdrop, using the ::backdrop pseudo-element:

#logo_popover::backdrop {
  background: rgb(0 0 0 / 75%);
}

Step 4: doing this in WordPress

If you want to do this in WordPress, you can add it with a simple bit of code, for instance, like this, to your wp_footer.

/**
 * Output a popover for the logo.
 *
 * @link /popover-contextmenu/
 */
function joost_logo_popover_footer() {
?>
	<div popover="manual" id="logo_popover">
		<h3>Looking for our logo?</h3>
		<p>You are in the right place.<br/>Check out <a href="/logo/">our logos & style page</a>.</p>
		<button popovertarget="logo_popover" popovertargetaction="hide">Close</button>
	</div>

	<script>
	document.querySelector(".is-logo-image").addEventListener('contextmenu', function(event) {
		event.preventDefault();
		document.querySelector("#logo_popover").showPopover();
	}, false);
	</script>
<?php
}

add_action( 'wp_footer', 'joost_logo_popover_footer' );

Docusign to Documenso

joost@joost.blog (Joost de Valk) — Fri, 15 Mar 2024 00:00:00 GMT

Migrating from SaaS to open-source

In my quest to move most of my SaaS services to self-hosted open-source software, I’ve taken on the next step: our document signing infrastructure. Everyone seems to be using Docusign, but as with all SaaS tools, that can get really expensive. While we don’t do a ton of contracts at Emilia Capital, I did already spend $25 / month, just to get some PDFs signed. Documenso is a great solution!

Documenso has basically all the features Docusign has with one exception so far: I’ve not been able to get it to do initials yet, which might get annoying for some document types. But otherwise it already looks pretty feature complete to me for a document signing solution.

Migrating from SaaS to open-source
The software
The benefits
The economics
- The cost of Docusign
- The cost of Documenso
Conclusion

The software

Installing Documenso is relatively trivial if you know how reverse proxies work, but let’s walk through the steps;

You follow the install steps for self-hosting in their readme; it requires npm and a bunch of other libraries on your system and you need to have a Postgres database running.

A few things I found: I didn’t need the marketing site, so I removed --filter=@documenso/marketing from the start command in package.json.

Also, make sure to not add trailing slashes to the URLs in your .env file, that caused me a few minutes of debugging.

You’ll probably want to disable registration, so set NEXT_PUBLIC_DISABLE_SIGNUP to true.
Make sure to install the next cli on your server, like so:

npm install --global next

Make sure to run Documenso as a service. On my Debian install that means copying their example file to /etc/systemd/system/documenso.service, modifying the values to be correct for my system, and then enabling the service like so:

systemctl enable documenso.service

Assuming you’re using Apache, create a virtual host file for the (sub-)domain you’re installing Documenso on, and make it look something like this, assuming you’re running Documenso on port 3000 (if not, just change it in your service file that we created in the previous step and here). This assumes you’re doing SSL with Let’s encrypt / certbot.

<VirtualHost *:443>
	ServerAdmin webmaster@localhost
	DocumentRoot /var/www/sign.example.com
	ServerName sign.example.com

	ErrorLog ${APACHE_LOG_DIR}/sign.example.com-error.log
	CustomLog ${APACHE_LOG_DIR}/sign.example.com-access.log combined

	SSLEngine on
	SSLCertificateFile /etc/letsencrypt/live/sign.example.com/fullchain.pem
	SSLCertificateKeyFile /etc/letsencrypt/live/sign.example.com/privkey.pem

	ProxyRequests Off
	ProxyPreserveHost On
	ProxyVia Full
	ProxyPass / http://localhost:3000/
	ProxyPassReverse / http://localhost:3000/
</VirtualHost>

Email sending

For the email sending part, Documenso allows you to put several different types of services into your .env file. I use Postmark for everything, so I used it here too, with just its SMTP sending service.

Once you’ve done all that, you should have something that looks like this:

## The benefits

Next to the cost benefits, more on that below, there are a few other benefits:

You know where your data is and where it’s hosted. If you’re a European company, you should probably set this up on a European server and save yourself a lot of hassle with legal.
You can more easily, without cost restraints, add team members.
If you’re building another app, the idea for Documenso is that you should be able to integrate it into your apps. Check out their site, it’s nice stuff!

The economics

Setting this up took me about 30 minutes, so from a time perspective it wasn’t a deep investment.

The cost of Docusign

Docusign was costing me $25 / month, which I’ve always found annoying for just getting some PDFs signed. It was also the reason to have only one account, which meant that I was doing some manual work myself because I owned the Docusign account that I didn’t hand off to a colleague. I probably should’ve never done that and just paid for another account, but the cost right now was $300 / year.

The cost of Documenso

Zero. Hosting cost: if I’d taken on a separate server for this: ~$5 a month. I didn’t though, so for us it’s no extra cost.

Conclusion

Documenso is a great tool to have for your company. If you have someone who can set it up for you, it’s a great addition to your companies toolbelt without having to pay for Docusign.

Read my previous from SaaS to open-source guide: from HelpScout to Freescout.

HelpScout to Freescout

joost@joost.blog (Joost de Valk) — Thu, 07 Mar 2024 00:00:00 GMT

Migrating from SaaS to open-source

Everyone uses a lot of SaaS applications to manage their day-to-day business. So much that the cost of it becomes prohibitive. I’ve decided that I’m going to try to move as much of the software we use to run Emilia Capital from SaaS to open-source and self-hosted solutions as possible. I will document every step because this might help others do the same.

The first step I took was moving from HelpScout to Freescout. Email is still a very important process for us, as both the legal processes around investing and the invoicing processes of all our businesses are still tied to email. So, we were using HelpScout to manage that. When I started looking for an open-source alternative, Freescout immediately turned out to be a very solid system that was also incredibly easy to install. Since it looks incredibly similar, the switch was easy on our team.

## The software

Freescout is based on a traditional PHP + MySQL stack, with either NGINX or Apache in front of it. The install instructions are simple, because installing it really isn’t hard if you’ve ever done some work on a server. There are also packages for installers like Softaculous and Fantastico. You’ll need to ensure you have all their required PHP modules, but the installer makes it easy to see which modules you still need.

I set Freescout up using Postmarkapp as an SMTP server for the outgoing email because I don’t want to use Sendmail or something like that on my server and consider email reliability super important. In reality, you can probably set that up fairly easily too.

For the incoming mail, I used simple free Gmail accounts. Here there’s an important trick to know: first, create those accounts. Then, enable 2-factor authentication and create an application password in your Google settings (go to 2-factor authentication in your Google account and scroll down). Then go into Gmail for that account and enable IMAP. Then use that application password to log in to your Gmail account from Freescout, and everything should work seamlessly.

Modules

I added the following modules to our setup for now:

API & Webhooks (needed for the migration)
Extended Attachments
Reports
Saved replies
Tags
Workflows

The benefits

There are a few benefits to self-hosting Freescout that might not be immediately obvious. Of course, there are economic advantages; more on that below. There are also legal advantages: all the data is on your premises. No need to tell users that you’re using a third-party service for support interactions.

I think the biggest benefit for us as a company is the liberty to create more users and inboxes when we need them and not be bound by very restrictive account limitations or prohibitive costs. Which brings us to the economics:

The economics

The cost of HelpScout

I’ve been a HelpScout customer for over a decade, we were big users at Yoast, but their pricing started annoying me more and more. We were paying $80 per month for 4 standard user accounts. We needed more than two inboxes, but we decided against using more and changed our process a bit because the cost would otherwise have gone up to $160 a month.

So, over a year, the TCO (Total Cost of Ownership) of HelpScout would be 12 * $80 = $960. As our company grows a bit, that cost would go up. Had I used it the way I’m now using Freescout, I’d have 2 more users and more inboxes; the total cost per year would have been $2,880.

Cost of Freescout

For Freescout, since we host it on a server I already had running for other things, it’s free. But realistically, the monthly cost is like $5 for a Vultr server that can run this (that link gets you $100 credit to test Vultr and gives me some credit, too). On top of that, you need an email account to receive the emails. As I mentioned, I just used free Gmail accounts for that.

I did spend some money on modules for Freescout. They’re one-time costs, as they have lifetime licenses. The total cost of that was $67.94. The most expensive step was the data migration. You could probably do that yourself if you wanted to, but I used help-desk-migration.com to get it done in under an hour. Cost of that: $375.

ItemCostServer cost per year$60.00Modules – one-time fee$67.94Migration$375.00Total cost of the first year****$502.94Total cost second year$60.00As you can see, the ROI of changing to Freescout will be positive in the first year. In the 2nd year, this is a very big cost reduction.

Conclusion: success! What’s next?

I’ll call this one a win. I hope it inspires companies to do the same. I’ve got quite a few other services I’m considering for the next step, but I’m open to your suggestions: what should be next?

Playground Blueprint Builder

joost@joost.blog (Joost de Valk) — Wed, 06 Mar 2024 00:00:00 GMT

I have been playing with the WordPress Playground a lot. As I did, I wanted a way to quickly make a copy of an existing site to a new Playground. It seems an ideal way to test new plugins, themes, and other things you’d want to change.

You do that by making a Blueprint with all the data for the site you want to mimic. So, I started working on a Blueprint Builder. It’s far from done and shouldn’t be run on a live site yet. But it has all the concepts of something that could be very useful. It makes a blueprint of the current site’s plugins, themes, and options. It even makes a WXR URL available through the REST API to download a site’s content into your Playground.

If your site is relatively simple, it works (if I haven’t just broken it with my last commit). If you have a custom theme, it doesn’t play nicely yet, because it doesn’t download that (yet). It also ignores any plugins it doesn’t recognize from WordPress.org.

I told Adam Zielinski about it (he created Playground). He asked me to open-source it, as it could perhaps be used within the project. Similar ideas existed there as well, which I think is exciting. So I did, and the code is now available to all.

Cleaning up WordPress option clutter

joost@joost.blog (Joost de Valk) — Thu, 29 Feb 2024 00:00:00 GMT

Note: this plugin has received a ton of updates since its inception. It’s now been moved to Progress Planner, where the team there (including myself) are developing it further. So please check out AAA Option Optimizer on Progress Planner’s website.

If you run a WordPress site for a while, it’ll become cluttered. You’ve installed and uninstalled dozens of plugins over the course of your site being alive, and many of those plugins have left their footprint (their options) in your database. I’ve come up with a solution for this, which is the first step of something that I think can be even nicer.

Why is this a problem?

This wouldn’t be so bad if WordPress didn’t load all the options that are set to “autoload=yes” into its memory in one big load action. Since stray options stay behind in the database when you uninstall a plugin, those options are still loaded into the memory on every pageview. Every. Single. Time.

How can we solve this?

To solve this, we have to follow a few steps:

Determine how bad the problem is by seeing how many autoloaded options we have and how much memory is being used by those options right now.
For a while, keep track of all the autoloaded options and whether they are being used. Simply put: every time an autoloaded option is used during page load, we store it. Then we compare the list of options that have been used with those that have been autoloaded. The difference between the two are all the options that are autoloaded but never used.
Now we have a list that we can go through and decide to either delete the option entirely or, if you’re unsure, simply remove the autoload=yes from it. If you do the latter, everything keeps on working; the option just isn’t loaded on every page load.

You’re now asking yourself: that sounds quite simple, but… How do I do that?? Well. I’ve built a plugin for it 🙂

Meet AAA Option Optimizer

This plugin is called AAA Option Optimizer. What’s with the funny name? Well, WordPress loads plugins alphabetically. To best measure which options are being loaded, it’s best for the plugin to be loaded early. So… I called it AAA Option Optimizer!

When you install the plugin, it measures the current standings immediately and stores them. No need for you to worry about that. Then you should go and click around your site. Be sure to hit every page, both in the front end and in the admin. You could use something like Screaming Frog on the front end.

Then you go to the plugin’s page. It’s under Tools. It’ll look something like this:

On this page, you remove the options that you no longer need, and you can optimize away. Be sure to make a backup before you do this, because you might of course break things. This is a power tool, don’t give it to people who don’t know what to do with it.

Future versions: auto optimization?

I actually think this could be turned into something that auto-optimizes. Removes autoload from options automatically when they’re never used, add autoload to options that get called a lot but aren’t auto-loaded.

I’d love to hear your thoughts!

Update 15th April 2024: the plugin made it to WordPress.org today!

Designing plugins to be uninstalled

joost@joost.blog (Joost de Valk) — Fri, 02 Feb 2024 00:00:00 GMT

I feel a particular class of plugins in the WordPress world should be more prominent. Plugins that help you do something, which you then delete when you’re done. There are a lot of these, and they’re super useful. I’d probably call them utility plugins.

Let me explain with an example: say you’ve been bitten by the Tags bug and have too many tags. (This is something I’ve been publishing about on the Fewer Tags site). You have too many tags; you need to do an audit, merge some of them, delete some, and you’re done. Fewer Tags Pro can help you do that. And then, because of how it’s designed, you can remove it! This is a superb thing for site owners because it means you don’t clutter your site. Not yet another plugin you have to update every other week, but it does help you perform your task faster.

Designing for uninstallation

It’s not something plugin developers do often: designing their plugins to be uninstalled. In the case of Fewer Tags Pro, it was clear that this plugin needed to create redirects. But that doesn’t mean I want it to do redirects. That would mean building a redirect maintenance system and other annoying things they already have. I don’t want this plugin to add yet another one of those things. So, instead, it integrates with Yoast SEO Premium and Redirection. If you use either one, it will create the redirects in their redirect system. This means the plugin can be safely uninstalled.

Aside: this did mean those plugins needed to have APIs that allow integration. To be fair, both could have been better in that case, but I managed.

There are quite a few plugins like this, and we’ve probably all used some of them:

Migration plugins like All in one WP migration;
The famous regenerate thumbnails plugin,
Find my Blocks, awesome to find where you’re using blocks you’re trying to get rid of

But you know what’s funny? Two of the above plugins have more than a million installs. Why are these plugins staying installed on sites? Because we don’t have an active system saying: “Hey, this is a type of plugin that should maybe not be on your site for months.” Maybe we should change that. Maybe we should add a flag in the plugin header or the readme that means: “This plugin is meant to be used for a short period.” We could then warn a site owner if a plugin like that has been installed on their site for more than a month and urge them to uninstall it.

Other metrics

We could give the developers of these plugins something in return if they set that flag! We could give them stats on the number of activations and de-activations they had on WordPress.org. They’d get this data instead of (or maybe next to) data on the number of active installations. That data is not normally available to plugin developers and would actually show these developers how often their plugin is being used.

What do you think? Good idea? Suggestions for improvements?

Research: WordPress publications misuse tags

joost@joost.blog (Joost de Valk) — Tue, 30 Jan 2024 00:00:00 GMT

This post was migrated to this site in January 2025 when we took down the Fewer Tags project site.

Tags are not used correctly in WordPress. Approximately two-thirds of WordPress websites using tags are using (way) too many tags. This has significant consequences for a site’s chances in the search engines – especially if the site is large. WordPress websites use too many tags, often forget to display them on their site, and the tag pages do not contain any unique content. In this post, I’ll discuss these three main tag problems I’ve diagnosed WordPress websites with and propose solutions.

About the research

As an SEO consultant, I have repeatedly seen the problems that tags and tag pages can create. To better understand the extent of the problem, I’ve conducted a small research project. I’ve investigated 30 sites and analyzed the way these sites dealt with tags and tagpages. My analysis included 10 popular WordPress-based mom blogs, 10 WordPress news blogs (based on WordPress and writing about WordPress), and 10 randomly chosen WordPress-based publications from the WordPress showcase.

The results shocked me. Mostly because I was expecting quite a few people to do it wrong; I wasn’t prepared for how wrong it would be. One site had 3-4,000 posts and 30,000 tags. Another had 1,000+ tag pages that all didn’t work. 87.5% of the sites that used tags had one or multiple tags with just one post in them.

For each of these sites, I checked the following:

If they used tags.
If they linked these tags from their posts.
If they had a sensible number of tags compared to the size of their content library.
If they had tag pages with just one post in them.
If their tag pages actually worked.
If their tag pages had a description/intro copy.

First problem: way too many tags

People often add tags to their posts because WordPress suggests they do so. Perhaps people also add tags because they’re used to adding tags on other platforms, such as Instagram (you shouldn’t do that). Sometimes (in about 10% of the websites in our research), websites even end up with more tags than posts. And that does not make any sense. Tags are supposed to structure your content. A good use of tags means that the number of tags on a website is significantly smaller than the number of posts. In our research, we’ve treated sites with 5 posts per tag on average as having a good use of tags. In Figure 1, you can see that the majority of the sites that we analyzed do not smartly use tags.

![Pie chart showing that:

37.5% of sites have a good use of tags 20.8% of sites have 3-4 posts / tag 25% of sites have 1-2 posts per tag 16.7% of sites have more tags than posts](./images/use-of-tags-on-sites-1-png.webp)Figure 1: The (over-)use of tags on sites.People are often unaware that WordPress creates a specific new URL (a tag archive page) for each new tag you add to a page. WordPress lists all the articles with that specific tag on that tag page. But we see a lot of cases in which writers add a tag only once or twice. Of the sites we’ve analyzed, 87,5 % have tags containing only one post. Such a tag page will only list that one post. This page is very empty, and that’s bad for your chances of ranking in the search engines. It’s also a horrible user experience: you click from the article to the tag archive, because you want to read more content like that, and you’re presented with one choice: the article you just read. That’s a disappointment.

Figure 2: Percentage of sites with tags with just one post.## Second problem: Tags are not displayed

The second problem with tags is that the tags aren’t even displayed on websites. I’ve encountered this problem in about one-third of the sites I analyzed. People add tags to a post before publishing, but their website’s theme (design) doesn’t show these tags. In many themes or designs, there is just no place for the tags.

Figure 3: Percentage of sites using tags that don’t show those tags to the user.Having tags and not displaying them means you’re users are not using those tags at all. You’ll create a bunch of empty tag pages that nobody can visit. But since the tag pages are created and listed in your site’s XML sitemap, search engines will visit them, index them, etc. This takes resources away from your site pages that you want them to visit and index.

Third problem: Tag pages do not have descriptions

The third problem with tags has to do with thin tag pages. Each new tag that people add to a post on their site will create a new tag page. This tag page could be where visitors to your website will find more information about a certain topic. It should at least have a description or an introduction. However, my research shows that only one WordPress site has done some optimization for its tag pages.

If you create a tag page, it would make sense to spend some time optimizing it. A tag page could be much more than just a list of links pointing toward articles with a tag. Ideally, A tag page should be a kind of homepage for a specific topic. It should help your visitors figure out and navigate to the content they want. However, the standard WordPress backend only allows for simple text editing on the tag pages. It provides an HTML field, which requires you to write code even if you want to add a simple thing like an internal link. This could be a reason why tag pages on most WordPress sites are so very thin.

Figure 4: Percentage of sites that don’t have a description on their tag archives.## How do we solve these problems?

We should use fewer tags, we should always display the tags we are using, and we should draft engaging introductions to our tag pages. That’s the solution. But that can be a lot of work.

Only recently have I developed a quick solution with the Fewer Tags plugin. The Fewer Tags plugin (free) will help you automatically remove all tag pages with fewer than 10 posts. You can also alter that number to something else. It’s a straightforward and quick way to handle the problem immediately. However, installing the Fewer Tags plugin does not offer a sustainable solution for dealing with tags.

Fewer Tags Pro, the premium version of the plugin I’ve built, will help you to work towards a more sustainable solution. The Pro plugin helps you to merge and redirect tags. Next to that, buying Fewer Tags Pro gives users access to an online course, explaining in-depth everything people need to know and understand about tags. We’ll help you display tags on your site and draft engaging introductions to those tag archive pages. The premise of Fewer Tags Pro is that we help you solve all the problems with tags and use tags to your advantage.

Several sites (6 of the 30) in our analysis do not use tags. Within the category of WordPress websites, this percentage was especially high. We suspect these sites are aware of the problem with tags and decided to use only categories as a taxonomy. That makes sense and would also be a solution to the tag problem.

How did I do my research?

Conveniently, because all these sites run WordPress, they have pretty recognizable patterns. If they used an SEO plugin, I checked their XML sitemaps. If they didn’t, or I couldn’t find tags in those, I used the REST API to get the number of posts and tags on their site using these scripts. If that didn’t work, I ran a crawl, which I luckily only had to do on one site. After that, I randomly opened some articles on each site to see if those had tags displayed in them.

When I found tag pages, I clicked on them and checked what they looked like, and I also clicked on articles on that tag page to see if they linked back to the tag page I just came from.

To determine whether a site had too many tags, I looked at their XML sitemaps again or the REST API using the scripts linked before. Mostly because those are the easiest to check. I then divided their number of posts by their number of tags to determine a posts-to-tags ratio.

Why these checks?

Now, let me explain why I checked these particular things one by one:

Why check if they use tags?

Mostly because I wanted to see how many sites use tags and didn’t want to exclude sites that didn’t use tags. There’s nothing wrong with not using tags.

What’s bad about not linking to tags?

If you’re not linking your posts to your tags, then what are you using them for? If you’re just using them for internal reasons, the best thing you can do is not show them to visitors anywhere. If they’d done that, I wouldn’t have been able to find their tag pages.

When does a site have too many tags?

Tags should bring content together. They should allow you to go from one piece of content to another and meaningfully connect pages. If you have too many tags, this doesn’t help the user’s navigation, and you actually also add tons of URLs on your site that search engines have to index, so that’s not helpful.

I determined a site had too many tags when the ratio of the number of posts divided by the number of tags was bigger than or equal to 5. So if you have 200 tags and 1,000 posts, I’d still deem that “acceptable”, even though I personally think that’s a bit much already.

What’s bad about one post in a tag?

You might be thinking: what’s bad about having one post in a tag? Well, let me explain. You get to a post. You liked that post so much that at the end of the post, you click on a tag underneath it, because you want to read more of that type of content. You are what we call an engaged reader. Then, the site presents you with a tag page that has just one post in it. The post you’ve just read. That’s a disappointment, right? That’s what we’d like to prevent.

What do you mean “if their tag pages work?”

On a few of the sites, the tag pages didn’t work. They were either empty (not listing the articles with the tag) or had big missing items, like missing a main heading on the page, or had clear errors. Tag pages are not pages people look at every day. That became clear.

Why does a tag need a description?

If someone clicks on a tag, they want to read more about a certain topic. With a description, you could help them. We explain how you could do this in our course about tags.

Multi-currency awesomeness with EDD & Cloudflare

joost@joost.blog (Joost de Valk) — Fri, 26 Jan 2024 00:00:00 GMT

We’re getting ready at Emilia Projects (which is the “run our own projects part” of Emilia Capital) to sell our first plugin, the Pro version of Fewer Tags. We will sell that using Easy Digital Downloads (EDD), which I have not used for years, so I’ve been re-learning how to work with it. One of the things I’d found they had added since I started using it is a very nice multi-currency plugin.

The plugin works mostly by adding ?currency=<currency> to the URL and allowing people to switch using that, but… I don’t necessarily want people to switch. We have two currencies, Euros (EUR) and US dollars (USD) and I want to give everyone in Europe EUR and everyone else USD. So, the default currency of the site is USD, and we needed to find a way to make people in the EU default to EUR. That’s where Cloudflare comes in!

I started with an approach using cookies and a worker, but this has the downside of not allowing Cloudflare caching to work properly, as the page differs depending on where you come from, but the URL doesn’t change. Thankfully Jono pointed this out to me when I was discussing it with him, and he pointed me towards transform rules.

Cloudflare transform rules

Cloudflare has added a very nifty feature called transform rules that allow you to change URLs based on a number of variables. One of them is the country that people are visiting your site from. So, I created a transform rule that adds ?currency=EUR to every URL on the site for visitors from the EU (and GB). Here’s the “request match” part of that rule:

Clicking this together in Cloudflare can be some work, so here’s the full expression. You can just put that in by clicking “Edit expression”:

(
  ip.geoip.country in { "AT" "BE" "BG" "CY" "CZ" "DE" "DK" "EE" 
   "ES" "FI" "FR" "GR" "HR" "HU" "IE" "IT" "LT" "LU" "LV" "MT" "NL" 
   "PL" "PT" "RO" "SE" "SI" "SK" "GB" }
  and not starts_with( http.request.uri.path, "/wp-" ) 
  and not http.request.uri.query contains "edd_action"
)

The first bit is simple: target everyone in these countries. The second “And” ensures you can still use your WordPress admin properly, and Cloudflare doesn’t add the query string we’re about to add to JavaScript that might be loaded, images, etc. The third “And” ensures you can still add and remove items from your cart and do other things. You might need to adjust this a bit for your site if you have other query parameters specific to your site.

And here’s the “Then” part of this rule:

What’s probably not immediately clear to you but is super valuable: the user does not see that ?currency=EUR is secretly being added to the URL! It only gets added in the communication between CloudFlare and your server and not towards the user. So, the cache is different because the URL that CloudFlare is requesting is different, but the user sees the exact same URL as his friend in the US! This isn’t just great for the user but also for your SEO and your caching.

Displaying the right price

Now, the next challenge is displaying the right price and the right currency! Luckily, EDD has a shortcode for that: [edd_price id=<ID of download>]; this will output the price correctly. So, you need to drop the habit of mentioning price straight in your copy and instead use that shortcode everywhere. And yes, you can use that shortcode within blocks, for instance, buttons. Doing this in the editor:

Will show this on the front of the site:

Or this if you’re from the US:

## You don’t need a currency switcher!

By doing this, I would argue you do not need a currency switcher dropdown or buttons on your EDD site, and you can give people the correct currency, depending on where they come from, without any coding!

Props: thanks to my colleague Ari Stathopoulos for helping me with some of these bits and to Jono Alderson for his advice!

GitHub Actions to keep your WordPress plugin healthy

joost@joost.blog (Joost de Valk) — Tue, 23 Jan 2024 00:00:00 GMT

I absolutely love GitHub Actions and have been using them more and more for several purposes. As I’ve seen on some of the GitHub repositories I’ve been looking through, not everybody has as many nice use cases as we have in our repositories, so I thought I’d share some of my personal favorites.

<div class="not-prose rounded-lg border-2 border-primary/20 bg-tertiary px-6 py-5 dark:border-accent/20 dark:bg-stone-900"> <p class="mb-2 text-lg font-semibold text-primary dark:text-accent">Want your AI agent to do this for you?</p> <p class="text-base leading-relaxed text-stone-700 dark:text-stone-300">Install my <a href="https://github.com/jdevalk/skills?tab=readme-ov-file#-wordpress-github-actions" class="text-primary underline decoration-1 underline-offset-2 hover:no-underline dark:text-accent">WordPress GitHub Actions skill</a>, or point your AI coding agent at this article. The skill analyzes your plugin's structure and drops in the workflows it needs.</p> </div>

PHP

Composer diff

It can be quite hard to see what has been changed in Composer files, especially the lock file, which makes reviewing changes to them in pull requests hard. When added to your repository, this composer diff action adds a comment to every pull request that touches the composer.lock file and gives a human-readable table of changes to the composer file, as in the screenshot below.

### Composer security check

Another composer.lock scanning action, this (quite popular) Github action aptly called “The PHP Security checker” scans your composer.lock file for security issues and tells you when you should be fixing them. You can see how we’ve implemented it, for instance, in our Fewer Tags plugin here.

Code style & linting

If you’re coding, you’re making mistakes. Everyone does. But if you’re coding and you’re not doing everything you can to prevent those mistakes, you’re doing it wrong. That’s where code style & linting, and a few more tools come in. The first thing you should be running against your plugin is the WordPress Coding Standards. The name can lead you to easily misjudge this package. It doesn’t “just” judge that you’ve used the right number of spaces and the right type of naming conventions. It also tells you when you’re using functions that are discouraged, when you’re forgetting to add nonce checks even though you’re dealing with request data, when you’re not sanitizing or escaping strings before output, etc. WPCS is the first thing I run against every plugin I need to evaluate, whether for investment reasons or because I want to run it in one of our own projects.

To run WPCS on every pull request and commit is actually fairly simple. Here’s a good example implementation.

Another thing you should be running on every commit and pull request is a linter, both for your JavaScript and your PHP. Examples here (PHP) and here (JS).

JavaScript

Package-lock.json & Yarn.lock diff

These actions will make a human-readable version of changes to your package-lock.json or yarn.lock file, which can otherwise be quite opaque. For Yarn, you can use this action. I would recommend a setup like this, where it only runs if your yarn.lock actually changes. Similarly, you can use this action to create a human-readable comment for your package-lock.json file, with a setup like this example to only run when the package-lock.json file has actually been changed.

## Testing

PHPUnit

If you have unit tests (you should!!), you can, of course, run them. Feel free to copy the implementation from our Fewer Tags plugin on how we run these.

Manual testing

For very simple manual testing, I recommend using the WordPress playground (which I recently wrote about). I’ve created a very simple workflow action that leaves a comment on every pull request, with a link that opens the zip of that pull request in the Playground. This only works for plugins without a build process, so if you have an autoloader, it should work without that build, or you should adapt the workflow. But it’s proven very useful for quick testing of changes without having to spin up local environments or do anything else, really. It looks like this:

It automatically updates the comment after each commit on a pull request, to the latest commit.

WordPress.org deploy

I’m too lazy to do all the manual Subversion work needed to release plugins; with 10up’s WordPress deploy action, you don’t need to do that ever again. Combine it with a .distignore file (example) to really create clean deploys to WordPress.org while also having all your WordPress.org assets (headers, icons, screenshots) in a nice .wordpress-org directory. In some of the implementations we have, like this Comment Hacks one, it even creates a release on GitHub automatically. All I have to do is merge a branch into my main branch, tag it with the right version number and commit and push that tag to GitHub, and it does all the work from there.

Share! What are your favorite actions?

I’d love to hear in the comments what your favorite GitHub actions are.

WordPress needs more dev-rel

joost@joost.blog (Joost de Valk) — Mon, 01 Jan 2024 00:00:00 GMT

The WordPress ecosystem is very good at talking to each other. We do so on platforms like the official WordPress Slack, Post Status, Twitter, and many other places. We are, however, not as good at keeping up with the rest of the open-source world – something Marieke recently mentioned in her column too – and I think this is where we are missing many opportunities. In this post, I explain why WordPress needs more developer relations (dev-rel) and what we could do to fix it.

I’ve recently been diving into a fascinating ecosystem of new open-source startups. I started by looking at the investment portfolio of OSS Capital, which Matt pointed me to. Automattic is an investor in them / partners with them; I don’t know the exact details. It has many interesting startups: apps like Cal.com, a Calendly alternative, which also allows you to self-host. Or OpenStatus, an open-source monitoring & status page system. Or Formbricks, an open-source surveying platform. All these companies link to each other, talk about each other, they have “OSS friends” pages on those sites telling their users about each other. But what bothers me: none of them talk about WordPress.

Some of them do WordPress a bit. For instance, Cal.com made a WordPress plugin and then promptly forgot about it and never updated it again. And honestly: that’s a shame. It’s a shame not just for them but also for the WordPress ecosystem. And it happens because we have no one actively maintaining those relations and telling them: “Hey, you could have a ton of users for your platform if you made it a bit more easily accessible to WordPress users.” (While we’re at it, we could also help them set up better blogs. It’s apparently not fancy to use WordPress for those blogs (they mostly use Next.js), but the result is that most of their blogs are horrible.)

So let’s fix WordPress developer relations!

The fix to this problem is actually incredibly simple. We don’t need one individual to do this. We need to all do this together. The power of WordPress is in the number of people we have! If you look at new open-source projects (and you should, I think a lot of these have the potential to save you thousands of dollars a year on SaaS fees), check whether they have WordPress integrations. If they don’t, create an issue on their GitHub. These people think and talk like “us” in many ways, and they also spend a lot of time on GitHub like every WordPress contributor does, so use that to your advantage!

I’ll do this myself; I’ve already made an issue on Cal.com’s plugin repository to update their plugin and create a Gutenberg block. There are many of us; together, we could really reach out to all those new open-source tools and help them and WordPress thrive!

Clickable WhatsApp links

joost@joost.blog (Joost de Valk) — Sat, 30 Dec 2023 00:00:00 GMT

The simple things make life easy. One of them is WhatsApp. And more specifically: customer service through WhatsApp. I love it when companies offer WhatsApp support. What I love a little bit less is when they offer it, but they make me jump through hoops to get to it. I recently ordered some shirts from SuitSupply and ended up needing their support. This is what I ran into:

The email address wasn’t linked and thus not clickable, nor were the WhatsApp numbers. Talking about making me work for it… Now, I don’t even need to know your WhatsApp number. I just need you to give me a link that works. And this stuff ain’t hard, so here comes the explanation:

How to create Clickable WhatsApp links

WhatsApp makes linking to a chat on WhatsApp super simple by allowing you to link to their wa.me short domain. So, instead of showing that number in full, making it all look super complicated, just do the following:

<a href="https://wa.me/31655110516">Talk to us on WhatsApp</a>

Leave out all the spaces, dashes, and pluses. You can even add a text to start with:

<a href="https://wa.me/31655110516?text=I%20need%20help%20with">Talk to us on WhatsApp</a>

On their official support page, WhatsApp also provide official “chat on WhatsApp” buttons, but I’d probably just use a text link myself. That’s it! Easy does it.

And then it could look something like this:

Now, of course, they really should underline their links too, but that’s another matter.

Post Scriptum: other links

So, you really also should link all email addresses with mailto: links and phone numbers with tel: links, so I can click them and start writing that email or start calling you:

<a href="tel:0031655110516">Call us</a>

or:

<a href="mailto:service@suitsupply.com">Mail us</a>

It’s not just WhatsApp links, everything that you expect me to do on a phone, you should be making it easier for me, and adding links like this is super simple.

Good-looking GitHub profile pages

joost@joost.blog (Joost de Valk) — Wed, 20 Dec 2023 00:00:00 GMT

After my blog post about healthy GitHub repositories, I learned that many people didn’t know how to create .github repositories and what you can do with those. That automatically also means that you don’t know about organization profile pages, so I wrote a quick post about those, too, which you’re reading now. I’ll start by explaining how to make a good-looking organization page, and then we’ll talk about your personal GitHub profile.

<div class="not-prose rounded-lg border-2 border-primary/20 bg-tertiary px-6 py-5 dark:border-accent/20 dark:bg-stone-900"> <p class="mb-2 text-lg font-semibold text-primary dark:text-accent">Want your AI agent to do this for you?</p> <p class="text-base leading-relaxed text-stone-700 dark:text-stone-300">Install my <a href="https://github.com/jdevalk/skills?tab=readme-ov-file#-github-profile-optimizer" class="text-primary underline decoration-1 underline-offset-2 hover:no-underline dark:text-accent">GitHub Profile Optimizer skill</a>, or point your AI coding agent at this article. The skill reviews your personal or organization profile and generates an optimized profile README.</p> </div>

A good-looking GitHub organization page

The Emilia Capital GitHub organization profile is very new and a bit empty, but it already looks quite good:

### 1. Add a profile readme

The fact that this looks good is the result of a few things. First, and most importantly, it’s the profile/README.md in our .github repository, which renders above our pinned repositories. This allows you to explain who you, as an organization, are and point to interesting links. It’s really as simple as this: create a .github repository, which you should for reasons I outlined in my previous post anyway, add a profile directory and create a README.md file in that directory.

2. Pin your most interesting repositories

Of course, ensure those repositories have good descriptions, so people know why they should check them out. In practice, a simple first step is pinning your most starred repositories.

3. Add your company URL and social profiles

Go to your organization settings:

Click your Avatar
Go to “Your Organizations”
Click “Settings” for your organization.

Add your company’s URL, a description, and a few social profiles like your organization’s X/Twitter and LinkedIn. Then, make sure to verify that organization URL by going to “Verified & approved domains” in the left-hand menu and following the steps outlined there.

A good-looking personal GitHub profile

You can quite simply have a good-looking personal GitHub profile like mine:

### 1. Create a profile readme

Just like you can create a .github repository for organizations, you can create a “magic” repository for your personal profile too. But this repository has a different name: it should be identical to your username. So for me, it’s jdevalk, as you can see here. If you add a README.md file to that personal repository, you get the same benefits as the profile/README.md does for organization profiles. And then it renders above your pinned profiles as you can see in the screenshot.

Note: you can have a .github repository for your personal GitHub too. It has all the benefits of a .github repository for your personal repositories; the only thing that’s different is the method for the profile page.

2. Pin some repositories

Here too, you should pin some favorite repositories, and make sure those repositories have proper descriptions. Of course, those repositories should be healthy GitHub repositories themselves too.

3. Set your profile data

Go to your profile settings and add your social URLs, website, description, etc. Make sure to do this as it “connects” this repository to your important social profiles and then, create a link back from those social profiles and your website to your GitHub profile too. This makes people trust your profile a lot more.

How to create a healthy GitHub repository

joost@joost.blog (Joost de Valk) — Tue, 19 Dec 2023 00:00:00 GMT

As a result of investing in a fair few companies and getting a lot more requests to invest, I look at GitHub repositories a lot. I often run into GitHub repositories that don’t look well maintained. So, I thought I’d write a small article on what a healthy GitHub repository should look like.

<div class="not-prose rounded-lg border-2 border-primary/20 bg-tertiary px-6 py-5 dark:border-accent/20 dark:bg-stone-900"> <p class="mb-2 text-lg font-semibold text-primary dark:text-accent">Want your AI agent to do this for you?</p> <p class="text-base leading-relaxed text-stone-700 dark:text-stone-300">Install my <a href="https://github.com/jdevalk/skills?tab=readme-ov-file#-github-repo-optimizer" class="text-primary underline decoration-1 underline-offset-2 hover:no-underline dark:text-accent">GitHub Repo Optimizer skill</a>, or point your AI coding agent at this article. The skill audits your repo against the checklist below and generates drop-in files for anything missing.</p> </div>

In this post

Repository description and URL
README file
Community health files
- Hack: a .github repository
Tags and releases
Branches
Conclusion

Repository description and URL

The repository description should not be empty. If you go to your repository and see “No description provided”, it looks like you didn’t care. Click the cog, add a line of text describing the repository, and you’re done.

If you haven’t set a URL for your project, add it now, too.

README file

A good README should, in my opinion, cover a few things. Let me discuss them in the order I believe they should appear in.

Action badges

Showing your default actions, like Code Style, Linting, Unit tests, etc., is a good idea. It shows me you care. GitHub makes this incredibly easy, so not doing it is a no-go. If you don’t have any actions and this repository is for code, you should wonder why that is. Do you really think you don’t need linting or a code style? Think again.

What does this thing do?

No long prose is needed here, but one or two paragraphs that explain what the repository I’m looking at is for. It could be similar or the same as the description, but probably slightly longer.

Installation instructions

If your GitHub repository is public, you say, “Look at my/our code and contribute!” Of course, as a lover of open-source software, I love that. However, how to install your software, plugin, theme, or whatever it might be is often unclear. Your README.md should start with, or very quickly, point me to clear and complete installation instructions. What do I do after I check out your code? Do I need to build anything? Please don’t make me read your code to determine your build steps.

How do I contribute?

The “How do I contribute” section can be tiny, as it should point to your CONTRIBUTING.md file, which we discuss below. A special note, either here or under its own heading, should be made about security issues. This should probably point to your security policy, which should live in your SECURITY.md file.

Your readme.txt is not your README.md

WordPress plugins and themes come with a readme.txt, that has specific requirements. Your README.md file should be different. And yes, that means you need both, but I think your README.md doesn’t need to be shipped with your plugin or theme; it should describe (how to use/interact with) your repository, which is subtly but distinctly different from your built plugin or theme.

Community health files

GitHub allows for a fair few different community health files. I consider having these the absolute minimum:

CONTRIBUTING.md
This file should specify how to contribute, which code style to follow, and what to do and not do when doing pull requests or creating issues. Of course, you should have pull request templates and issue templates to guide these processes as well.
SECURITY.md
This file should first clarify where to report security issues and how quickly you expect to respond. It may contain more info, but this is the most basic requirement.
CODE_OF_CONDUCT.md
It’s smart to have a code of conduct. It’s easy to point to when you get unwanted behavior and also provides you with a guideline on how to handle problems. You don’t have to develop one yourself; I’d suggest using the amazing contributor covenant.

Hack: a .github repository

If you create a .github repository for your organization, you get to set up “default” health files for all your repositories in one go. See the Emilia Capital .github repo as an example. This lets you set up default issue and pull request templates for all your repositories as well, which is super convenient.

Tags and releases

If you release software, I expect you to tag those releases in your repository, so it’s easy to find them, preferably with downloadable zips and a proper changelog. Far too often, I run into WordPress plugins that don’t do this and make life more complicated than it needs to be.

It’s smart to add a .gitattributes file to your repository, one of the things you can do with that file is define which files should be export-ignore, so they’re not in your zip files.

Branches

If you have dozens or more stale branches on your GitHub repository, I will know (or assume) that you’re not in control. Clean them up. Delete no longer necessary branches, set a reminder for yourself to do this every so often, and make sure that you’re in control of all of them.

Conclusion

Clean up! This is your workspace, but most importantly, the space where you either foster collaboration or shun it.

Be sure to check out the sequel to this post: good-looking GitHub profile pages.

Plugin demos with the WordPress playground

joost@joost.blog (Joost de Valk) — Thu, 14 Dec 2023 00:00:00 GMT

For years, plugin developers have been looking for ways to demo their plugins to users. Users don’t read readme.txt files or plugin pages on WordPress.org. They want to try and see a plugin in action for a bit, which is a much better way of seeing whether a plugin solves your problem. The new WordPress playground is a perfect way of doing that. So, the meta team at WordPress.org started building a preview button. While the first iteration was not ideal, a few weeks ago, @Tellyworth posted on Make WordPress about the revisited Playground experiment. I, of course, started playing with this feature, and now I can’t wait for it to go live for all users.

First, let me show you what I’ve created. Click this link to open the Fewer Tags playground in a new tab.

As you’ll see, this opens a WordPress Playground, with Fewer Tags installed, like in the screenshot above. But it does more. It brings you to the correct page for you to see what Fewer Tags does (the edit tags page), and it loads a notification that explains what you can do there. This notification only loads and shows on Playground sites.

Let me explain to you how to do this:

Define a blueprint file

In the assets directory of your WordPress plugin’s subversion checkout, create a subfolder called blueprints. In that folder, create a file called blueprint.json. You can keep this file incredibly simple. The file for Fewer Tags is here, and it reads just this:

{
	"landingPage": "\/wp-admin\/edit-tags.php?taxonomy=post_tag",
	"steps": [
		{
			"step": "login",
			"username": "admin",
			"password": "password"
		},
        {
			"step": "defineWpConfigConsts",
			"consts": {
			  "IS_PLAYGROUND_PREVIEW": true
			}
		}
	]
}

As you can see, all it does is define our landing page, define a constant we’ll use below, and log the user in so they can skip that step. Everything else is done with another bit of magic.

Loading a Playground specific file

In the main file of the plugin, we load a class specifically for the Playground, like this:

// Detect if we're running on the playground, if so, load our playground specific class.
if ( defined( 'IS_PLAYGROUND_PREVIEW' ) && IS_PLAYGROUND_PREVIEW ) {
	$playground = new FewerTags\Playground();
	$playground->register_hooks();
}

And then we have a specific class that’s loaded, which takes care of a few things:

It generates some sample data, in this case, 2 tags and 20 posts.
It creates a notice on the page we’re demoing on, with a tiny bit of explanation of what you can do here.

Of course, this is a relatively simple plugin and, thus, also a fairly simple demo setup, but you can make this demo setup as complex as you want. Because this file is part of your plugin, you can have all the text translated on WordPress.org. If you use autoloading, as you should, the file never loads if you’re not on the Playground, so it’s not a burden to performance.

I think you’ll now agree with me that this is a game-changer for the WordPress plugin repository.

Saying goodbye to Marieke at Yoast

joost@joost.blog (Joost de Valk) — Mon, 26 Jun 2023 00:00:00 GMT

Last Friday (June 23rd 2023) we had Marieke‘s goodbye party from Yoast. With that, we’ve both completely stepped out at both Yoast and Newfold Digital, and can start to focus on new stuff. I wrote a speech for the occasion and thought it’d be a shame to not share it with the wider world:

When Marieke says goodbye to Yoast, I of course can not stay silent and so I have prepared this speech. To you all, but especially, to you, Marieke. As you and I, Marieke, started investing in other businesses and helping them along their road, I often reflect back on what we did at Yoast, and how we did things. And every time I’m amazed by the incredible impact so many of your achievements, or maybe we can even call them inventions, have had.

Of course, Marieke came up with the readability feature, and she created Yoast Academy. What was typical is how she did it: she had a clear vision of what she wanted to reach, and then started doing it: she did research, hiring some of the best people I’ve ever worked with in the process to help bring her ideas to life. It’s insane to realize now that when we started talking about readability, no one in the SEO space was talking about it. Now every SEO tool worth their salt measures readability to some extent. There were more features of course, for instance Marieke’s continuous focus on internal linking and site structure was also super important for those features.

She also created much of the branding, not the logo and colors, but the ideas around YoastCon, the magazines, everything around how she wanted people to perceive Yoast. She masterminded so much of that, that I don’t think I can ever do it all justice in a short speech here.

Equally as important as those features and the branding of Yoast was, was what she did in creating our company culture. From the very beginning, Marieke was involved in the hiring process. While she usually stays away from the paperwork, something she happily leaves to me or someone else, she had very specific ideas about how she wanted our company to work. Some of those ideas she brought from her own previous work experience, but others were entirely new and I think true innovations.

The premise of her HR policy was extremely simple and effective: be good to your employees and they’ll be good to you. She made it a point to know not just everyone’s name (which became hard enough already), but also know their partner’s names and their kids’ names.

She empowered women at Yoast, with her Empowerwoment project, which I think we can at least partly credit for Yoast having such a good & diverse leadership team for so long. The core values project we did, which had the entire company align on “why do we do what we do?”, something that seems obvious when you’re a startup but becomes increasingly hard when a company grows. She was also so clearly the more subtle one: instead of, like me, just asking “is it done already?” all the time, she got everyone to “hup hup”, or: move faster.

And then during COVID, Marieke started doing daily vlogs for the entire company. Pouring energy into the company in a way that I can still not really understand, with so much energy being required at home too. That time was tough and really brought to light the enormous amount of pressure that comes with being responsible for so many people, both users of our software and families relying on their income from us. That’s when we decided: this is no longer “it” for us, we need to sell.

So: we did. We sold Yoast, that process was a journey in itself, and since then, we’ve had our struggles, as leaving a company you’ve founded is probably always going to be hard. But I am really proud of what we did together, and I think, Marieke, you are too.

In many ways, you, Marieke were the only one of us who really innately understood & felt the needs of a growing company. I’m often referred to as the sole founder of Yoast, and whilst that’s probably technically true, I do prefer, and want to hereby state unequivocally: without Marieke there as my co-founder, and yes, I think you should call yourself co-founder of Yoast, this company would not have been what it is today. And for that, I want to thank you, Marieke. You’re the best colleague I’ve ever had, and I look forward to working with you on much, much more, including, next to all our exciting investments, our next own new project, a new WordPress plugin, which has the best working title ever: Mareekee. We hope to get to the point where you need Yoast and Mareekee for every WordPress site.

I love you.

How to deal with plugin security issues

joost@joost.blog (Joost de Valk) — Tue, 11 Apr 2023 00:00:00 GMT

Dealing with security issues for your WordPress plugin can be hard and even a bit scary, certainly when it’s happening to you for the first time. In this post I try to outline all the steps you should take as a plugin developer. So, let’s dive in. This isn’t a short post, so here’s a table of contents:

Having plugin security issues reported to you
Fix the issue
- How fast should you fix security issues?
Tell your users about your plugin security issue
- Changelog
- Other communication
Don’t be too hard on yourself

Having plugin security issues reported to you

You probably use a platform like GitHub, or even the WordPress.org forums to get bugs reported to you. These are public channels which is super helpful for most of your bug reports. However, this is not good for security issues. This is one of the reasons security issues in open source are a bit harder. You’re probably not used to secrecy, but where it comes to plugin security issues, you do need a bit of secrecy sometimes. So, you need a way for security researchers to contact you. And trust me, if your plugin gets popular, they will contact you.

You have a couple of options where it comes to receiving security reports. The simplest option is to set up an email address like security@ on whichever domain it is you use, or a contact form. Know though that you’ll get a lot of reports that you have to vet and not all of them are that useful. You also will find some security researchers will require you to sign emails with keys, functionality not offered by many email providers, which can be a hassle to set up.

A better way (that’s relatively new) is using Patchstack’s – completely free!! – vulnerability disclosure program. They deal with validating the issue and only bother you when you actually have to fix something, with a very detailed report. They’ll deal with the security researcher, getting them to follow ethical disclosure policies, and because they’re a CNA (a “CVE numbering authority”), they can assign a so called CVE, a number by which a security issue can be referenced later. If they’re handling your vulnerabilities, it immediately gets you into the process of properly assigning CVEs. I think it’s completely awesome that they offer this service for free, and I’d recommend everyone to use it. I use it for my own plugins, see for instance the VDP page for my Comment Hacks plugin.

“Advertising” how people can report issues

You should mention how people can report plugin security issues to you in at least a few of these places:

On the plugin’s webpage / website.
On the plugin’s GitHub page (preferably through a security policy Security.md file).
In the plugin’s readme.txt and thus on the WordPress.org plugin page.

Make sure that you regularly test your contact method. Trust me, you don’t want to miss these emails. They’re important.

Encouraging security reports

If you make money from your WordPress plugin(s), directly or indirectly, I’d highly suggest setting up a bug bounty program (examples: WordPress core’s bug bounty program on HackerOne, Yoast’s bug bountry program, Elementor’s on Bugcrowd). It’ll encourage security researchers to look at your code a bit more. This might feel counterintuitive, because you don’t want security issues, but the fact is that security issues are a fact of life. It’s best to admit that you’re human and benefit from other people scrutinizing your code. Being proactive about security is the only way to make sure that you don’t find yourself feeling incredibly bad because lots of sites were hacked through your plugin.

Learning from others

When other plugins have security issues, you don’t gloat. You don’t think you’re better, trust me, you’re not. Everyone has security issues. What you do is you look at the report, and see what it says. Then you think and check whether your own code would be susceptible to such an attack vector as well. If it is, you patch it.

Fix the issue

Once you’ve identified the problem in your code, you fix it. Of course. Then you go through your code and search for other places where the same thing happens. Then you check your other plugins, if you have any, and check for the same issue. Once you’re sure you’ve fixed all the instances of this particular issue, you submit the update to the plugin repository. I’m not going to go into the technical details of coding securely in this post, others can do that much better. But I will say this: having a pull request open on your public repository with words like “fix for security issue” in the description is probably not the best idea. This is where you have to keep things a bit of a secret until you’ve actually fixed the issue.

Security researchers are often surprisingly helpful in testing your fixes. Communicate with them. If you do have a company like Patchstack vetting your reports, make sure to stay in touch with them as well.

How fast should you fix security issues?

I’m frequently baffled when I hear how long people take to fix plugin security issues. I honestly think most vetted security issues for plugins should be fixable within 48 hours. Literally 48 hours from receiving the report, I’d expect a good plugin developer / plugin development team to have shipped the fix. Depending on how severe the issue is, it could be ok to wait a few days if it happens to come in on a weekend. It’s never OK to wait for weeks. As your software gets bigger, this might get a bit harder. WordPress bundles its core security releases together, I’m not part of that process but I know that that can take a bit longer. But if you’re one small group of people, just make sure to get that security release out, it’s better for everyone.

Tell your users about your plugin security issue

The hardest part of every security issues is always: how do we tell our users? I know what you’re thinking “they may not trust us anymore”, “what if they switch to another plugin?”, etc. Trust me, I know those thoughts deeply, I’ve been there. And trust me when I say: the truth always prevails. So, don’t hide your security issues. Be open about them. Be confident enough to say: “We’ve been told about this issue. We’ve fixed it. You can rely on us to fix our security issues in a responsible manner. Please update to the latest version.” Users will reward that behavior over time. So, let’s be clear about where we need to communicate:

Changelog

The most important bit first: of course your fix should be listed in your changelog, which should be in your readme.txt and it should clearly be identified as a security fix. If someone reported the issue to you, clearly credit the reporter (if they want to be credited). This again validates that you’re happy to receive security reports. You might want to use the Upgrade notice functionality in the readme.txt standard as well.

Other communication

Pushing the update out and mentioning it in your readme.txt is not enough if the issue is severe. If people’s sites might get hacked because of the issue, you’d better tell them, through Twitter, your newsletter, whatever channel it is you have, that you’ve released a security update.

Don’t be too hard on yourself

I know that I’ve taken security issues a bit too personal myself in the past. Security issues are a fact of life. Of course you need to think about security as you develop. Of course people can expect some level of basic security “posture”. But beyond that, security issues happen. Good developers / product owners stand out by how well they deal with them. I’d be very wary of any developer that says he never has security issues.

If you have questions, feel free to ask them in the comments. If you need help communicating about a security release, I’m @joostdevalk on the WordPress Slack, and I’m always happy to help.

Thanks to Oliver Sild from Patchstack and Marius Jensen for their feedback on early drafts of this post. Photo by Jaye Haych on Unsplash.

The plugin developers guide to…

When you build a WordPress plugin, you obviously have to write code. But then, when you release it to the world, especially when you release it open source, there are things you have to do that you didn’t know about before. Things that don’t really show up in code tutorials. I’ve decided that I should share my experience in building one of the most popular plugins in the WordPress world in a series of posts, like this one about plugin security issues.

If you have topics you’d like me to cover in this series, comment on this post or email me and let me know!

Gravity Forms notification routing with a lookup table

joost@joost.blog (Joost de Valk) — Thu, 16 Feb 2023 00:00:00 GMT

I had a need for complex routing of notification messages in Gravity Forms. The email could, depending on a dropdown field, go to some 50-60 different recipients. So I needed a custom lookup table with notification recipients and a way to look up the recipient based on that dropdown field, without exposing all those recipient emails to the outside world. It turned out to be fairly simple once I knew how to approach it, but I thought I’d blog it to make it easier for everyone.

Backstory

For my local football club AWC, where I’m a volunteer, teams need to be emailed about upcoming games sometimes. Messages range from changing the date of a match, to telling the team that the game has been cancelled, etc. Historically, there’d be one email address for that and someone would manually forward those messages to the right team. As the club currently has some 55 teams, that’s becoming quite a bit of work. So I wanted to see if I could make that a bit simpler.

The setup

For this setup I’ve used my trusted forms plugin for the last decade, Gravity Forms, plus a few other plugins:

GP Populate Anything – one of many very useful plugins from GravityWiz.
Advanced Custom Fields – my go to plugin when I need to create custom fields.
Custom Post Type UI – to lazily create a custom post type.

Now let’s go through the different steps I took for this:

The notification recipients

First, I created a post type contact-persons with “supports” set to false (or “none” in Custom Post Type UI), which means that it’ll not have a post editor etc. I then created a field group in Advanced Custom Fields (ACF), with 4 fields: the team name, the contact’s first and last name and an email address. I made sure that field group only showed on the contact-persons custom post type:

Now when I edit a contact person, it looks like this:

I created a custom plugin for this setup, which has two functions. One takes care of the form routing, the other one takes care of saving the team name to the post_title field. I could have left the Post title on the page, but I feel this is actually a better look for the form in terms of usability. This code is quite simple:

/**
  * Sets the team name as the title of the post type for team-contacts.
  *
  * @see 'save_post'
  *
  * @param int     $post_id The post being saved.
  * @param WP_Post $post.   The post object.
  */
function joost_set_team_name_as_title( $post_id, $post ) {
    // If this is a revision, get real post ID.
    $parent_id = wp_is_post_revision( $post_id );

    if ( false !== $parent_id ) {
        $post_id = $parent_id;
    }

	if ( $post->post_type !== 'team-contacts' ) {
		return;
	}
	
    // Unhook this function so it doesn't loop infinitely.
    remove_action( 'save_post', 'joost_set_team_name_as_title' );

    // Update the post, which calls save_post again.
  	$team_name = get_field( 'team_naam', $post_id );
	wp_update_post( [ 'ID' => $post_id, 'post_title' => $team_name ] );

    // Re-hook this function.
    add_action( 'save_post', 'joost_set_team_name_as_title' );
}
add_action( 'save_post', 'joost_set_team_name_as_title', 10, 2 );

The form

I built a contact form in Gravity Forms, just the way one normally would, and added on field to the very top: Which team do you want to send this message to? This field is a drop down, which I set to “Populate choices dynamically”, using the GP Populate Anything plugin. That field then takes the teams from the newly created custom post type and lists them here:

As value, I kept the Post ID, because we’re going to use that in our second custom function, which does the routing. I hooked this to the generic gform_notification filter and decided to check if the notification name matched what I used. You can also use a more specific filter, but this was easy enough for my use case:

/**
 * Filter the notification recipient to the right team, if this is a team notification.
 *
 * @param array $notification The notification array.
 * @param array $form         Not used.
 * @param array $entry        The entry array.
 *
 * @return array The notification array.
 */
function joost_notification_emails_team( $notification, $form, $entry ) {
    // Only change notification to address for team notifications.
    if ( $notification['name'] == 'Team notification' ) {
 
        $field_id = 1; // The ID of the team select field.
 		$post_id  = rgar( $entry, $field_id );
		
        $notification['to'] = get_field( 'email', $post_id );
    }
 
    return $notification;
}

add_filter( 'gform_notification', 'joost_notification_emails_team', 10, 3 );

The notification

Now we need to create our notification. It’s as simple as creating a notification that looks the way you want it to. You can add any email in the To field, as it’ll be replaced by the code above anyway. The only thing that’s important is that you give the notification the same name as you’ve used in the code above.

That’s it! Now you’ve got a working team contact form, where if you add teams, they show up in the drop down and the right contact person receives the email based on your “lookup table” post type. Notification routing done right!

Bonus points

Disable Custom Post Type UI

One of the things I absolutely love about the Custom Post Type UI setting is its export functionality. Under CPT UI → Tools → Get Code, it gives you the function to register your post type. Copy paste that code to your custom plugin and voila: you can disable Custom Post Type UI.

Create a better overview page

On the overview page, you’d see the team name and the date it was published. Not very useful. We can change that to: team name, contact name, contact email, and remove the date, with the following custom code:

/**
 * Register columns for the edit contact persons overview page.
 * 
 * @param array $columns The existing columns.
 *
 * @return array The filtered columns.
 */ 
function joost_add_team_contact_columns( $columns ) {
	$columns['contact-name']  = 'Contact name';
	$columns['contact-email'] = 'Email';

	unset( $columns['date'] );

	return $columns;
}

// Register the columns, specifically for our custom post type.
add_filter( 'manage_team-contacts_posts_columns', 'joost_add_team_contact_columns' );

/**
 * Output the column values.
 * 
 * @param string $column_name The name of the column.
 * @param int    $post_id     The post ID.
 */
function joost_team_contacts_column_content( $column_name, $post_id ) {
	if ( $column_name === 'contact-name' ) {
		echo get_field( 'voornaam', $post_id ) . ' ' . get_field( 'achternaam', $post_id );
	}
	if ( $column_name === 'contact-email' ) {
		printf( '<a href="mailto:%1$s">%1$s</a>', get_field( 'email', $post_id ) );
	}
}

// Handle the value for each of the new columns, only on this post type. 
add_action( 'manage_team-contacts_posts_custom_column', 'joost_team_contacts_column_content', 10, 2 );

So, now we have a Gravity form with notifications routing, a management interface for teams and their contact persons and as you add teams or change contacts, the form updates dynamically and keeps sending notifications to the right people. Thanks to the various people in Post Status‘ #club channel that helped out with their suggestions!

WordPress’ admin UI needs to be better

joost@joost.blog (Joost de Valk) — Wed, 25 Jan 2023 00:00:00 GMT

The WordPress’ admin UI needs to be drastically improved. It should be improved not just for WordPress core itself, but it should implement a simple and clearly defined open design system, so that plugins and themes can use it to build their own interfaces. Now that Yoast’s new Settings UI is out in the open for everyone, it’s also clear for everyone that we’ve deviated from what we used to do, which was to use the “standard” WordPress admin UI components. The reason for that is simple: WordPress’ admin components are old fashioned and haven’t progressed as they should. We simply can’t build a modern interface with them. It saddens me enormously though, that we’ve had to take this step.

The problem with us choosing our own path, is exactly that: we’re choosing our own path. Every plugin is choosing their own path. For every plugin, users have to learn new UI elements and experience a new UX. Instead of building upon a common UI library, where of course we might change some colors and use our own logos, but we’d all use the same buttons, input fields, etc.

Every OS needs a design system

I think that if WordPress wants to be the Operating System of the web, as has been claimed by its project leader Matt Mullenweg, it should take a leaf out of the playbook of desktop OS’s. Microsoft has its Fluent Design System, Apple has its Human Interface Guidelines and tons of design resources. Apple also hosts yearly, highly prestigious, Design Awards highlighting those who’ve innovated on top of their design system.

WordPress needs a design system and it needs it fast

It’s not that I think the WordPress core designers aren’t aware of this, or maybe even agreeing with it. There are issues that show that they want to move in the right direction. The last version of the design experiments plugin (which was a step in the right direction) was released more than 3 years ago. This issue, which has many of the needed components etc in it, is from July 2021, well over 1.5 years ago, and only about half way done. I think we really ought to speed up this design / development. Might be that there are simply not enough people working on this, or we aren’t prioritizing it right as a project.

The current state is simply bad: WordPress core basically has 3 designs now. The edit post page I’m typing this in looks nothing like the Posts overview page, which looks nothing like the Site Health page. And then you go into plugins and each has their own UI there too. This makes WordPress as a whole harder to use. This is how we lose CMS market share to companies like Wix and Shopify (who each do have their own design system).

Now, of course, we’ve open sourced our own UI library at Yoast. Of course we’ve shared it with our colleagues at Newfold Digital and I’ll certainly push for us to use it more across the wider company. Other plugin developers and theme developers shouldn’t need to do all that work again and can use it, and hopefully, contribute back.

I wish this had not been needed, but here we are. Maybe this post can help rally a few more people to work on WordPress core design, Despite all that, I’m super proud of what the new settings UI in Yoast looks like, so go check that out!

Social & Schema images: naming considerations

joost@joost.blog (Joost de Valk) — Wed, 06 Jul 2022 00:00:00 GMT

I’ve been playing a lot with Schema and Social images recently and one thing has become clear: we need better naming of these images and we should probably improve the Schema.org image standards a bit. In this post I want to briefly discuss the different needs and my proposed (very simple) naming scheme.

As I was looking at how different social platforms supported WebP, I quickly realized that while they’re all using images, they’re using them in very different ways. Shocking, I know.

The problem is that in Schema, we just have one image attribute on articles. Well, technically, we also have thumbnailUrl, but that’s not even close to descriptive enough.

Let’s go through the two main types of images:

Poster images

An “OpenGraph image” these days is often the main image of the post with text on it. The title of the post, maybe an author, a site’s name and/or logo. It’s similar to a poster you would create for a movie or a performance in say a theater. So, why not call it that: a “poster image” or short “poster”. There’s precedent of that in HTML actually, where the video HTML tag has a poster attribute that should be shown until the video has loaded.

Platforms like Facebook, Twitter and LinkedIn use these images when an article is shared on their timeline. Making these images be effective posters is very important to your click-through rate from these platforms.

Featured images

The main image of an article is called the “featured image” in WordPress and many other systems. For some platforms, like Google Discover, this image is much more suitable. Note that it doesn’t mean that these are all similar; this can be a good news image, a beautiful image of a travel destination or a great illustration. Important is that there’s no text on it, or at least, the text is not a main feature of the image.

If a platform is going to put text over the image or prominently display the title in its vicinity, it’s important that it grabs the featured image, not the poster image.

For example, compare this post’s featured image and post image:

This post’s featured image

This posts’s poster image

So what’s the problem?

Facebook, LinkedIn and Twitter all read OpenGraph metadata. So we can feed them a poster image in the meta og:image tag. Discover reads Schema, so we feed it an image in the Article schema, and we just feed it the regular featured image (behavior we changed recently in Yoast SEO, coming to you soon).

Pinterest historically read both. My thinking is that for Pinterest’s needs, it’d be much better to have the featured image.

But… I would like all social networks to start reading Schema at some point. I think we all want that, because the amount of metadata in a page right now just to say “hey use this image” is bordering on the ridiculous (and don’t even get me started about the platform’s ridiculous behavior around image sizes).

The solution

To be able to have both poster and featured images in Schema and let platforms pick the right one, we need more specificity in what type of image is what. My suggestion is to introduce an attribute poster-image and an attribute featured-image on the Article Schema, that would both take an ImageObject. This would resolve the ambiguity and make it possible for every platform to fully rely on Schema. In an even more perfect world, we could even add an array of ImageObjects in both, with different sizes for the different platforms.

Optimize crawling: let’s turn things around!

joost@joost.blog (Joost de Valk) — Thu, 02 Jun 2022 00:00:00 GMT

I wrote about crawl optimization last week, mostly about getting stuff that you don’t want crawled to not be crawled. There’s more to say about that, and I will in follow up posts, but first we need to talk about how to get the stuff that you do want indexed, crawled and indexed by search engines. To that end we’ll have to talk about XML sitemaps and the new kid in town, IndexNow.

At the end of this post I’ll propose a radical change to how we approach indexing. One that might not come into existence anytime soon, but that I’d love to discuss with everyone in the industry.

XML Sitemaps

XML sitemaps are meant to give search engines a list of all the URLs on a site. Ideally, they’d also include the images on those URLs and the last modification date. We’ve had XML sitemaps in Yoast SEO for years. In 2020, the Yoast team and Google team and some others collaborated on getting XML sitemaps in WordPress core. The WordPress core sitemaps are really just a list of URLs, the Yoast SEO ones are slightly more sophisticated.

All search engines use XML sitemaps, and almost all of them use the last modified date as well as the image extensions. Notable non-user of the last modified date? Google, as noted in this recent podcast. This honestly actually surprised me as that seemed like a bit of information that would be fairly reliable in a lot of cases. But this might actually have to do with the definition of a “change”, something I’ll talk about below.

IndexNow

A recent addition to the website toolkit of ways to inform search engines is IndexNow. It’s a fairly simple protocol that allows you to ping search engines a URL that has changed on your site, or a list of URLs that has changed, and they say they’ll spider them quickly.

This new standard is supported by Bing, Yandex and Seznam, so far, and I’ve heard rumors of other search engines joining their ranks. This might not be useful for everybody with this current set of search engines, but the idea in general isn’t a bad one, especially as they “echo” pings to each other. This means you only have to ping one endpoint and they’ll send them along to all. We’ll be adding support for it to Yoast SEO Premium soon and we look forward to seeing the impact it makes, if any.

What is a “change”?

The problem with notifications of changes is that the definition of a “change” is extremely cumbersome. If you change a single typo in a page, does that require a re-crawl? Probably not. Or was that typo in the title? That might change things.

Whether something is a big or a small change is actually better judged by a human than by code, for now. That’s why I’d suggest actually making this a toggle in your CMS of choice, where a “small change” would not change the last modified date, and thus not cause a ping.

Could we turn this model around?

So I’ve been thinking about this topic for years. And something bothers me. Search engine crawlers are built on a very old concept: they follow links from one page to another, assuming that sites themselves don’t know which content lives on them. This simply isn’t true anymore for lots and lots of sites. A site purely built on WordPress, which SEO is managed by a tool like Yoast SEO, knows exactly which content is on it and what should be crawled and indexed. Yet, on sites like yoast.com, search engines crawl about 10x more URLs than we know are relevant.

What if search engines only crawled URLs that we explicitly allowed?

So: what if we turned the model around? What if search engines only crawled URLs that we explicitly allowed? What if we created a robots directive that played together with XML sitemaps. One where I could add the following lines to my robots.txt:

Disallow: /Allow-Sitemap: /sitemap_index.xml

And this would allow crawlers to only crawl URLs that are in those XML sitemaps. Sites should then explicitly set Allow rules for their assets too, for instance /wp-content/ and the like. For most, if not all sites this would mean a drastic reduction in the amounts of URLs that are being crawled. It would also mean a huge boost in efficiency in terms of caching and would thus be a great improvement for the environment, as sites would use less resources.

How to deal with canonicalization?

Now, I know you’ll say “but people link to URLs with parameters, we need to canonicalize those”. Yes, I know. Let’s say someone links to:

https://example.com/example-post/?gclid=2

And the XML sitemap only has:

https://example.com/example-post/

In this case, search engines / crawlers should simply canonicalize all URLs to the longest match they can find in the XML sitemap. Google, Facebook, Twitter and many others create random parameters they add to URLs all the time, and in doing so, create “e-waste” of giant proportions. Site owners should not be burdened with this. Crawlers should simply stop crawling nonsense URLs.

What’s missing from this idea?

This is where I turn it to the search engines and the SEO community: what’s missing from this idea. What would make this not work? What could we do differently if a site could be relied upon to know about its URLs, instead of search engines having to guess? I’d love to hear from you, on Twitter, or in the comments below.

Optimize crawling, for the environment

joost@joost.blog (Joost de Valk) — Tue, 24 May 2022 00:00:00 GMT

Search engines rely on spiders / bots to crawl the web and find (new) content. Every time they find a URL, they crawl it and if it’s interesting to them, they’ll keep crawling it basically forever. The bigger your site, the more URLs you have, the more likely every individual URL is to be hit multiple times per day (or even per hour). Cloudflare’s Radar estimates that currently 32% of the traffic on the web is bots. That means that 32% of the energy used to serve websites is used by bots.

Search engines are super eager to crawl. They will crawl literally everything that looks like a URL to them. This means that every URL you create will be crawled, and therefore every URL you create has an impact. Let that sink in: every URL you add to a website’s source will be crawled. Let’s talk about what that means and how we can optimize crawling.

The cost of (too much) crawling

We rarely talk about the cost of crawling. It’s costing search engines money to crawl, of course, but it’s costing you money too. If 32% of your traffic comes from bots, reducing that traffic might mean you pay less for your hosting. Regardless of whether there is a direct connection to your hosting bill, there is a connection to the electricity your site is using. To reduce crawling means your site uses less electricity which in the end is a win for the environment. Hence: optimize crawling, for the environment!

Let’s have a look at what’s causing all this crawling to begin with. I’m going to assume you know how search engines work at a basic level, if you don’t, this video will help you!

To put it simply: search engines are crawling more than we want, because they’re spidering things that are not useful to users. How come they’re crawling more than we want? Because:

You have a lot more than one URL per page

The problem I’m going to describe is by no means just a WordPress problem. Almost every CMS has this problem to some extent, but let me explain it to you by way of a WordPress example.

When you publish a new post on a WordPress site, you get a URL. Let’s say we’ve just created this URL:

https://example.com/example-post/

What most people don’t realize is that WordPress automatically creates a lot of URLs “around” this post, and it links them all in the source. Let’s make a list:

An RSS feed for the comments: https://example.com/example-post/feed/
An oEmbed URL so the page can be embedded: https://example.com/wp-json/oembed/1.0/embed?url=https%3A%2F%2Fexample.com%2Fexample-post%2F
Another oEmbed URL in an XML format: https://example.com/wp-json/oembed/1.0/embed?url=https%3A%2F%2Fexample.com%2Fexample-post%2F&format=xml
When you open these oEmbed URLs, you find that there’s an embed URL hidden in them: https://example.com/example-post/embed/
A JSON version of the post in the WordPress REST API: https://example.com/wp-json/wp/v2/posts/123
A shortlink of the same post: https://example.com/?p=123
If you have date archives, you’ve now also created https://example.com/2022/05/24/ as a URL and https://example.com/2022/05/ etc. See this date archive on Matt’s blog for instance: https://ma.tt/2022/03/23/.
If you have threaded comments (on by default in WordPress) and you don’t have Yoast SEO, you’ll have a link like this: https://example.com/example-post/?replytocom=54321#respond for every comment on your post.

If you look at your server logs (unfortunately, very few people do that), you’ll see that each and every one of those URLs gets crawled. None of these URLs add extra information that wasn’t already on the initial URL. None of these URLs should be of interest to search engines. But they don’t know that, so they’ll just crawl them anyway. This means that instead of crawling one single URL, your post, they’ve now got at least 7 versions of that URL to crawl and they will.

Simply put, I think developers should consider this far more than they do:

Every URL found in HTML will be crawled.
Every URL you create and put in HTML therefore has a cost.

You should ask yourself: is the benefit this URL brings, worth that cost?

Other types of pages

This is not just true for posts and pages in WordPress of course. Tasks that seem very innocuous to the end user actually have far more impact than they realize. If you add a tag to a post, for example, and that tag has never been used before, you’ve now created a new URL on your site. In fact, because WordPress automatically makes RSS feeds for every tag, you’ve created two: example.com/tag/new-tag/ and example.com/tag/new-tag/feed/.

The list goes on:

Plugins making it worse

Some plugins then add tons of extra URLs to create specific functionality. For example Jetpack’s “Sharedaddy” functionality adds one for every platform / share method you enable:

https://example.com/example-post/?share=tumblr&nb=1
https://example.com/example-post/?share=twitter&nb=1
https://example.com/example-post/?share=facebook&nb=1
https://example.com/example-post/?share=email&nb=1

Etc. etc.

Let’s be clear: Jetpack is certainly not alone in doing this, in fact, they are nice enough to add nofollow attributes to those links which should keep crawlers from following them (surprise: it doesn’t). Many plugins create almost infinite amounts of extra URLs (like dates in calendars) leading to infinite crawl “spaces” for search engines, which leads to very heavy crawling with absolutely no benefit.

Is this just Google?

By no means is this just a Google problem. Bing, Yandex, Neeva, Seznam, Baidu and many, many others crawl your site. Every day. They might not send you any traffic, but they do crawl your site.

What should web developers do?

Web developers, including those working on WordPress core, really should stop adding URLs to the source that don’t absolutely need to be there. You should really consider whether links need to be there for discoverability. Or if the feature is really needed at all.

In WordPress’ case:

The REST API functions perfectly fine without links pointing to it from every page’s source.
Adding shortlinks for every site that hardly anyone uses: not needed.
Comment RSS feeds for every post? In my not so humble opinion, those should go the way of the dodo.

Removing those links from every post would make a very real impact on crawling on the web, and therefore on power consumption by sites.

What can you do to optimize crawling?

Everyone: block bots or slow them down

If you’re not getting any traffic from a search engine, but they are heavily crawling your site: block them or slow them down. Bing and Yandex both adhere to the crawl-delay directive, so you might add something like:

Crawl-delay: 30

So they will only crawl your site once every 30 seconds or so. Play with the number there to get to something your comfortable with. With this setting they can still crawl 2,880 pages on your site every day.

If you’re on WordPress

Yoast is adding features to Yoast SEO to remove the links to these URLs from the source. In some cases, we’ll allow disabling the functionality entirely. When those features come out, you should enable them (they won’t be on by default because we don’t want to inadvertently break anyone’s site).

Sites that do not use the REST API to fill their pages (so, the vast majority of sites) should probably add this to their robots.txt file:

Disallow: /wp-json/

This blocks 4 of the 7 URLs I mentioned above and saves a ton of unnecessary crawling. Unfortunately, because some sites rely on using that endpoint for their content, we can’t default to this.

If you’re not on WordPress

Start looking at your site’s server logs. Find the URLs that are being crawled by search engines that are not useful to end users. Determine how they’re found and then start removing them. To analyze your server logs you can use a tool like Screaming Frog’s Log file analyzer.

If you find any other patterns others should know about, share them in the comments, and let’s optimize crawling together!

Want to read more on this topic? I’ve written a follow up post: Optimize crawling, let’s turn things around!

WordPress’ market share is shrinking

joost@joost.blog (Joost de Valk) — Wed, 11 May 2022 00:00:00 GMT

There’s no more denying it: if you look at W3Techs CMS market share numbers, WordPress’ market share is shrinking, losing 0.4% market share since February. I don’t like to be or sound alarmist, so when I first noticed these numbers, I waited a bit to write about it. Shopify, the #2 CMS in my last CMS market share report, is also shrinking a bit, though they’ve only lost a single percentage point so far.

If WordPress is shrinking, something else must be growing, this is, after all, a zero sum game. The very clear winners at the moment are Wix and Squarespace. Some other notable growers are Adobe, which is more on the enterprise side, and Webflow. Note that Elementor, which I called out recently as being a major growth driver for WordPress, still seems to be growing. So WordPress is shrinking despite Elementor doing well.

WooCommerce, on the other hand, seems to be losing some market share too.

WordPress market share over the last months

Let’s look at the raw numbers. Mind you, as long as I’ve been looking at this data, WordPress’ market share has always gone up.

MonthMarket shareJan 202243.2%Feb 202243.3%Mar 202243.3%Apr 202243.0%May 202243.0%11 May 202242.9%## Are we comparing apples to apples?

We know that Alexa recently shut down, and W3Techs has always used Alexa to base their top 10 million sites on. I checked with W3Techs last week and their response was clear: the API is still functional and they’re still using that, so the market share data hasn’t changed from that perspective.

Are these the right numbers to look at?

So, I don’t think this is the best representation of “the web” that can be made. I’m hoping to work with the people at the HTTP archive to make an auto updating version of my CMS market share numbers. I’ve run the numbers manually today on the HTTP archive dataset and in the last full HTTP archive crawl, which was for April, WordPress’ market share was also shown to be shrinking compared to March. So we’ve verified the shrinkage in two different sources. Both sources however are based on the most popular sites, so whether new sites are being built on Wix, SquareSpace or WordPress? We simply don’t know.

What’s to blame for this?

It’s honestly impossible to look at these numbers and not think “what’s happening here, why is that?”. After looking at it for a while, I’m coming to this conclusion:

If you look at cwvtech.report you’ll see that in the last year, sites on Wix and Squarespace on average have improved their site speed more than WordPress sites. WordPress has a performance team now, and it has made some progress. But the reality is that it hasn’t really made big strides yet, and in my opinion, really should. Project leadership still seems unwilling to focus on performance though, which has to do with the next point:
WordPress’ full site editing project is not done yet. Anecdotally, more and more people are having a hard time deciding how to build their site on WordPress. Wix and Squarespace are simply way simpler tools to build a site. As they improve their SEO tooling, there’s less and less reason to switch over to WordPress.

WordPress is being out-“innovated”

Conclusion: I think WordPress, for the first time in a decade, is being out-“innovated”. Now I say “innovated” because Squarespace and Wix are not really doing anything that new. They’re just implementing best practices for both site speed and SEO. They are, however, rolling that out for all their users. So all of their users get better and better page speed performance and improved SEO. As a result more and more of their sites are doing well. This means the set of sites that W3Techs is measuring, changes. And thus the top 10 million sites in the world will have more Squarespace and Wix sites. Those sites are actually doing better than their WordPress competitors.

This isn’t just on the WordPress core team. WordPress hosts all over the world should be working harder to make their clients’ sites perform better too. In fact, most of them should invest in WordPress more given how much money they make from it. If they don’t, they might see their free ride go up in smoke as the hosted services win.

You see: it’s not that you can’t do this with WordPress. It’s very possible to make super fast sites on WordPress, that perform incredibly well SEO wise. The problem is that in aggregate, WordPress sites aren’t doing as well. Too much of what a website needs to do to do well, is left to the people building and running sites. People who don’t know what they don’t know. WordPress, which has “decisions, not options” as one of its philosophies, is not even close to opinionated enough when it comes to performance and SEO.

How to fix WordPress’ market share?

If WordPress wants to maintain its market share or better yet, grow it, it’ll have to get its act together. That means it should focus on the performance of these sites across the spectrums of site speed and SEO. The Full Site Editing project is simply taking far too long. That’s causing the rest of the platform to lag behind current web trends. Without a drastic change in this approach I think WordPress will continue to lose market share for the next few years.

This is not what I’d like to be seeing, of course. If you have data that proves me wrong I’d honestly love to see it.

Transitioning to a new role at Yoast

joost@joost.blog (Joost de Valk) — Fri, 18 Mar 2022 00:00:00 GMT

At the end of this month, I will transition to a different role at Yoast. No, I will not leave Yoast. But, I am going to pursue some other dreams outside of Yoast. It’s time to spread my wings! Let me explain a bit on why I am making this transition and what it will look like!

Why?

For somewhat longer than a decade, Yoast has been my entire (work) life. And I still love the company. I still love our product. I still feel closely connected with Yoast SEO. However, I have always had ambitions outside of Yoast. I just was not able to pursue those, because I owned Yoast and was responsible for so many users and employees.

However, I would love to learn new skills or improve other products. I would love to learn from other people and to help other companies. My ambitions outside of Yoast were my main personal reason for selling Yoast. I really was getting bored. After our sale to Newfold, Yoast is running rather smoothly without me interfering on a daily basis. And that means it’s now time to spread my wings.

What about Yoast?

I was the final decision maker and primary driver for the product strategy of Yoast SEO for a long time. I’m super proud of going from 0 to currently 12.5 million users. Over the last few years, we’ve built a team of people that have a very good feel for what that product should be. The day to day SEO decisions I leave in the capable hands of Jono Alderson, who’ll now be Yoast’s head of SEO. With Marieke leading strategy, Thijs leading the company, and so many other great colleagues still there, I’m certain they’ll do fine.

For Yoast, this is going to be a good thing. Me leaving the day to day operation opens up a lot of opportunities for other people. There are so many talented people that work at Yoast, that’ll now have so many more chances to grow. And, I’ll still be there to share my knowledge and help make sure the product evolves in a good direction. I’ll also still be advising Newfold on WordPress & SEO and I’ll definitely be cheering for both Yoast and all of wider Newfold in the years to come.

What am I going to do next?

As said, I will stay on board for one day every two weeks at Yoast. I will advise on product and SEO strategy. I am really looking forward to spreading my ideas without having to execute all of them.

Next to that, I’ll finally have room for other things. I’m still contemplating what things get me most excited. Marieke and I invested in a number of companies and I look forward to getting a bit more involved with them. But I am also sort of curious to see what comes up now, and what I come up with when I don’t have to do day-to-day stuff at Yoast. It is so liberating not knowing exactly what I will be doing next year!

Elementor: WordPress’ secret growth driver?

joost@joost.blog (Joost de Valk) — Mon, 13 Dec 2021 00:00:00 GMT

Is Elementor the secret behind WordPress’ growth in the last years? We know from my CMS market share analysis that WordPress has been growing fast. Could it be that a lot of that growth is actually caused by Elementor’s popularity?

After I published the sixth iteration of my CMS market share analysis last week, some interesting questions popped up in a couple of places. One of them was a comparison to Elementor’s growth. Now I honestly wasn’t aware that W3Techs tracked Elementor, but it does right here, apparently since last February. This helped me do a bit of analysis.

What is Elementor?

Elementor is a page builder. They refer to themselves as a “website builder”, which in my opinion is a bit unfair. Elementor needs WordPress to be able to build websites. Without it, it’s a page builder. It makes building websites with WordPress easier and faster, and it makes it possible to do that without knowing how to code. The latter is probably one of the main reasons why it’s seeing tremendous growth.

Elementor raised $15M in capital in February 2020 and it looks like it’s applying that capital to create enormous growth. They recently launched their hosted version, which makes it truly competitive with services like WordPress.com, Wix and Squarespace.

How is Elementor doing?

Let’s start with the basic raw numbers. Elementor’s market share growth looks like this for the last 11 months:

Elementor market share within the top 10 million sites of the world.As you can see, Elementor has almost doubled in 11 months time. They are showing incredible growth. Note that this is market share of the top 10 million sites in the world, all the caveats that apply to my CMS market share post apply here too.

How about other page builders?

W3Techs tracks some other page builders too, all since last February. The most important to note here is WP Bakery, which currently is in use by 6.6% of websites in their data. What I found interesting is that while WordPress has shown decent growth, WP Bakery basically has seen no growth. There are some smaller players, one of whom has seen some growth, that’s Beaver Builder. According to these stats, they now have 0.4% of the market.

Where’s WordPress’ growth without Elementor?

When you compare all these growth numbers, it immediately becomes apparent what’s happening: Elementor is taking a bigger and bigger part of the WordPress “pie”. In the graph below, I’ve shown the relative growth of the market share percentage of both WordPress & Elementor and compared it with the delta between them: WordPress without Elementor. When Elementor grows 0.1% and WordPress grows 0.1%, the delta is 0%. In a few recent months, WordPress without Elementor has actually shrunken.

Elementor sites cannot exist without WordPress, so they are tied to each other. But I think the conclusion is fair that of all those new sites being built with WordPress, a very large portion of them is being built with Elementor.

I’ll make sure to make Elementor a part of my CMS market share analysis going forward as it has proven itself to be an important factor.

Yoast joins Newfold

joost@joost.blog (Joost de Valk) — Thu, 12 Aug 2021 00:00:00 GMT

Update: Since this acquisition, I've written about how the WordPress admin UI needs to improve and eventually transitioned to a new role before leaving Yoast entirely.

Today we announced that Yoast will join Newfold Digital. Marieke and myself, as well as Omar, Chaya and Thijs have sold our shares. We will, however, all remain in our current roles, as will all of team Yoast.

Why this step?

Well, honestly, this was a combination of a couple of things. First of all, “it’s the season”. We’re in a time of mergers and acquisitions in the WordPress world, a time of consolidation in what is still one of the most rapidly growing parts of the web. Yoast had always been bootstrapped, it was only our own money in this business, and we’d grown to 140+ employees. This was and is something we’re super proud of, but it was also causing stress. From COVID to euro/dollar exchange rates to the always existing, nagging fear of “what if they just all stop buying tomorrow” and more. We felt now was a good time for us to get away from that stress and find a company that could provide the backing we needed.

Another part is more personal: I was a bit bored. I wanted to go faster, do more, do new stuff, but couldn’t anymore. I was trapped in our success. So I look forward to not being responsible for finance and that part of running a business and leave that to the much more capable people that Newfold will bring in to help our finance team and hope to find new things on my path to do within the Newfold family of brands.

Why Newfold Digital?

When you’re talking to companies in the WordPress community, you quickly find out that they’re not all alike. The team at Newfold stood out. Sharon is a very inspiring leader and the rest of their team just oozes enthusiasm. All of our conversations with them left us with more energy than we started them with, something that I can honestly tell you has not been common in a lot of these conversations.

What will change now?

Not all that much for now, as we work on getting Yoast integrated into the wider Newfold family first. I look forward to collaborating with the other WordPress core teams (notably the Bluehost team) and see if together we can be even more successful in helping WordPress grow. And after that…. Well you’ll see soon enough!

But first: champagne! 🥂

In many ways this does feel like we’ve finished the game. And immediately we’re starting it again, now at the next level of difficulty, but with the new-found confidence of having partners that’ll be with us on the ride. But before we do that, I’m going to pop open a bottle of champagne 😃

Organize your screenshots on MacOS

joost@joost.blog (Joost de Valk) — Thu, 22 Jul 2021 00:00:00 GMT

MacOS has great built-in screenshot functionality. The problem is that, by default, it saves screenshots to your Desktop, which in my case turned my Desktop into a mess. I fixed this a while ago and today realized that this wasn’t an obvious thing for everybody, so I quickly wrote this blog post.

Time needed: 2 minutes

Let me quickly teach you how to improve your screenshot workflow and keep all your screenshots organized a bit better:

Create a screenshots folderFor me, my Screenshots folder is a folder in my Documents folder, that syncs to my Google drive. I’ve also dragged it into my left-hand sidebar in the finder, for easy access.
Open the screenshots appEither press ⌘-Shift-5 or open the Screenshots app from the Applications/Utilities folder.
Set the screenshots folder as your screenshot destinationYou might have to select “Other location” and find it.
Easy access: stack in your dockIf you want to easily access your screenshots, drag the folder to your dock. If you right click it, you can select “View content as” and set it to “Fan”, and set “Sort by” to “Date Modified”, and you get something like this:
Automatic renamingRecently I’ve started using Keep it shot to automatically rename my screenshots.
Bonus: better screenshotsMy screenshots got even better and more useful when I started using CleanShot X (not getting paid for this). It changed some of these things a tiny bit but the setup with my Screenshots folder and the fanning stack is still what I use every day.

Save your screenshots on Google drive

If you want even better organization for your screenshots, you should sync your screenshot folder to Google drive. It’ll OCR all the image automatically, allowing you to search for the text on those images in your Google drive. It makes finding screenshots after a few weeks an absolute breeze.

Happy that your screenshots are now organized and want to see other MacOS productivity tips? Check out my productivity hacks category.

Building a trusted web, step 1.

joost@joost.blog (Joost de Valk) — Wed, 28 Oct 2020 00:00:00 GMT

One of the biggest long-term trust problems of the web is reliably figuring out who published something first. Who was first has deep implications for ownership, and for areas like citation and (Google) news rankings. Today we have submitted an issue to the Schema.org GitHub which proposes a solution to fix this using timestamps on the blockchain. I view this as a first step in fixing some of the inherent trust issues on the web.

Full disclosure: this is a problem I care about deeply, which is why Marieke and I invested in WordProof, a company that set out to fix this problem. We’re not just investing money, I’m actively advising the company and we’re actively collaborating on proposals like this one.

What is this timestamp proposal?

Let me take the content from the issue:

The point is to “timestamp” new or modified content as you publish it. You then add the information on that timestamp, including where the hash lives, into the markup of a page while optionally also offering the resource used to generate the hash. This way, the hash becomes verifiable and optionally will allow for going through previous editions of the content (also verifiable).

A hash in a blockchain transaction indisputably proves that content existed in a specific moment in time (“transparency”). Furthermore, to put a transaction in a blockchain, you use a private key to authenticate. With this private key you can prove that you were the one placing the timestamp (“accountability”).

Similarly, a timestamp could be added to a transaction in combination with a set of terms of service, to prove that a certain set of terms are valid for that transaction. There are numerous applications of this system; this proposal focuses on time-stamping content to explain the basic architecture and provide validating services with the data they need to be able to verify the records.

What is a timestamp?

If you don’t know what a timestamp is, or don’t understand any of the concepts this proposal talks about, this might move a bit fast for you.

In that case, take some time to read about timestamps. Bas van der Lans, founder of WordProof, is much better at explaining it than I am, so see this video:

For me personally, this is one of the first blockchain implementations that has me deeply excited for its possibilities. Fun fact is: turns out blockchain was invented in 1991 for the specific purpose of timestamping (source). The problems timestamping solves are very real, day to day problems of website owners, merchants etc. We can fix those problems.

I hope we’ll look back at this in a few years and see that this schema.org issue was historic. That this is where we started the evolution and started building a trusted web, on top of the open web. And we’re doing that with open source software, an open standard and an open agenda. Let’s build the trusted web, together!

How to get week numbers in your Mac menu bar

joost@joost.blog (Joost de Valk) — Fri, 24 Apr 2020 00:00:00 GMT

This post explains how to get the week number in your Mac’s menu bar. Simple enough, right? At some point, as you start planning ahead in basically anything, you’ll run into “week numbers”. Week numbers are weird in that they’re not in your head all the time. But you do need to look them up more than you want. I ran into this and decided: there must be a way to solve this. And there is.

On my Mac the menu bar looked liked this:

And what I wanted was for the week number to be next to the date, in the top right. So first I looked in the Mac’s date settings (System Preferences -> Date & Time), but unfortunately, it doesn’t allow for this. So I started Googling. Quickly I found this tool called Itsycal.

So, I set out to change this.

First, we install ItsycalItsycal is a nice, free, calendar integration in your menu bar, for me it looks like this when I open it, showing a nice outline of my next 3 days:
We configure Itsycal to show the week numberCopy the top settings, under “Menu Bar” from the screenshot below. The content of the field is E d MMM H:mmm / w
Make sure Itsycal starts on system bootTo make sure all of this isn’t gone when you reboot your Mac, check the box on the General tab next to “Launch at Login”:
The menu bar should updateAnd it should now look something like this, with the Week number nicely showing after the date and time, on the left:
Get rid of the default calendarOpen System Preferences and go to Date & Time, uncheck “Show date and time in menu bar”. This takes the default date & time out of the menu bar.
Move the date & time to the rightYou don’t have to do this if you’re fine with the date & time on the left of your icons, but personally I dislike that. Luckily it’s very simple to rearrange your menu bar. Just press & hold cmd on your keyboard and then drag the date & time to the right:

That’s it! Now you have the date in your menu bar, just the way you want it, with the week number.

Happy to have your week numbers showing now? Find other quick improvements like this: check out my productivity hacks.

Automatically convert WebP files to PNG

joost@joost.blog (Joost de Valk) — Sat, 08 Feb 2020 00:00:00 GMT

I love CloudFlare. It makes websites lots faster. One of the things it does is change PNG files to WebP automatically when your browser (f.i. Google Chrome) supports it. This is awesome, as they’re smaller and it thus loads faster.

However, sometimes I need to download an image file and I end up with a .webp file. My operating system can’t handle those. So I have two options: opening the image URL in Safari (which doesn’t support WebP yet) or to convert them to PNG.

I had this happen to me enough that I decided to find a better solution. I run a utility on my Mac called Hazel, by Noodlesoft. This utility helps me keep my Downloads and Trash folders clean, among other things, with rules I can define myself. I decided to add a new rule.

Prerequisites

To convert .webp files to .webp files, you need Google’s WebP libraries. On your Mac, the easiest way to do that is with Brew:

brew install webp

Converting from WebP to PNG

The command to convert a WebP file to PNG would look like this:

/usr/local/bin/dwebp -o target.webp input.webp

Adding the rule to Hazel

Now it’s time to open Hazel and add a rule:

Our rule is going to:

Find files in the Downloads folder that end in .webp.
Run a script to convert those .webp files to PNG.
Move the .webp file to the trash.
Display a notification to show that it’s done all this.

All this is pretty easy to configure in Hazel. What you’ll need is the following embedded script:

The script is /bin/bash as we need to strip the .webp extension with some bash magic, and replace it with .webp:

/usr/local/bin/dwebp -o "${1%%.*}.webp" $1

After setting up all the rules as explained above and copy pasting the script, just save the rule. You can save any WebP file to the Downloads folder (or whichever folder you decide to run this rule for) and it should be automatically converted to PNG!

Google’s robots changes, the web & the law

joost@joost.blog (Joost de Valk) — Thu, 12 Sep 2019 00:00:00 GMT

Google announced a few days ago that they would change the way rel="nofollow" works for them. They would start treating nofollow as a “hint” instead of a “directive”. This means they go from “oh you don’t want us to go into that room? ok” to “oh you don’t want us to go into that room? We’ll see about that”. Basically, they went from friendly neighbor to annoying parent real quick.

Now rel="nofollow" is something you’d attach to links on your site. So I could nofollow one link and not nofollow another. There is also a meta robots directive, used like this:

<meta name="robots" content="noindex,nofollow"/>

This would historically direct Google to not show that page in its index, and not follow its links. Now Gary Ilyes tweeted this last night:

1. There's no meta robots ugc and sponsored, it won't do anything if you add that.
2. Meta robots nofollow is a hint now, like rel-nofollow.
3. I'll update the docs tonight to say this explicitly.

— Gary 鯨理／경리 Illyes (so official, trust me) (@methode) September 12, 2019

Gary tweeting about rel nofollow and meta nofollow. #2 is important in this context: “meta robots nofollow is a hint now”When Gary says “meta robots nofollow is a hint now”, I become slightly nervous. Because if nofollow in a meta robots element is a hint, what is noindex? Do they now want to treat that as a hint? or as a directive? Turns out, noindex remains a directive:

https://web.archive.org/web/20190927174635/https://twitter.com/dannysullivan/status/1171769802354057218

But, even if Google now says “noindex will remain a directive”, won’t that lead to years and years of discussion? In my experience, even many experienced SEOs don’t always understand the difference between directives and hints, and think they’ve excluded something when they haven’t. This change will only make this worse.

Google unilaterally makes changes

My biggest gripe with this is that Google is making these changes unilaterally. Bing, Yandex, Baidu: all support rel="nofollow" and other search engines probably do too. The same is true for meta robots nofollow. I don’t think it’s a good idea when Google decides on its own that it changes the “laws” of the web.

Google were the ones to introduce rel="nofollow", which gives them some rights to change that “standard”. However, meta robots nofollow has been around since 1996. In fact, this part of the meta robots page made me chuckle:

robots can ignore your <META> tag. Especially malware robots that scan the web for security vulnerabilities, and email address harvesters used by spammers will pay no attention.

Taken from the Robots pages

Apparently, I should consider Googlebot malware from now on 😉

Real world implications

Let’s look at real world implications: a link in a comment on a WordPress site used to have rel="nofollow" added to it automatically. We’ll now have to change rel="nofollow" to rel="nofollow ugc". We can’t take out the nofollow, because other search engines don’t support the ugc part, but Google, with its market domination, will urge us to make that ugc change.

Now Google’s first reply to this will be “you don’t have to change anything if you don’t want to, we even said that in our post”. And they did:

There’s absolutely no need to change any nofollow links that you already have.

Google’s Danny Sullivan in their announcement blog post on the nofollow changes

I read this and chuckled. Obviously Google needs to read up on the murder of Thomas Becket. Because of Google’s market dominance, people will do anything to get into their favor. They can make changes like this and the web will follow. The real question here is: shouldn’t we have legislation that prevents them from making these changes unilaterally?

In fact I’d say it’s time to go one step further: the web needs to have true standards for this. Standards that are preferably turned into law by the European Union, the US and China. But I dream too much perhaps. They at least need to be standards. Standards that all non-malware crawlers, including Google, will adhere too.

Traveling to Peru

joost@joost.blog (Joost de Valk) — Mon, 17 Jun 2019 00:00:00 GMT

At the end of April, beginning of May, 2019, Marieke and I visited Peru with our kids. The main reason for going to Peru was to attend the wedding of two of our friends Jessie & Jorge. These friends also happen to own a travel agency that specializes in travel to Peru, and helped us out a lot during this trip which was nice.

I wanted to share some highlights from this trip as I feel it’s a waste not to share these photos with more people.

Cuzco

The first city we visited in Peru was Cuzco, which, with its 3400 meters above sea level, was literally a breathtaking experience. The city is lovely, lots of tourists but the locals are very friendly and there are some very nice sights to see in and around Cuzco.

Cuzco’s main square, the Plaza de Armas

During a tour around some sights in the surroundings of Cuzco, I snapped a few pics of the surroundings:

Tambomachay View from Puka Pukara Cuzco seen from Sacsayhuaman Sacsayhuaman

After a few days in Cuzco we did what’s called a two day Inca trail towards Machu Picchu. In reality, this means you walk the Inca trails for one day and then have the next day to visit and admire Machu Picchu. This one day hike was about 12 kilometers and, according to my watch, about 240 full stairs. You go through some old Incan towns like Wiñay Wayna, which was incredibly beautiful.

Walking the Inca trail Beautiful, beautiful trails Beautiful nature during the trails Wiñay Wayna Wiñay Wayna

After that day of walking (it was heavy but doable) you end up on one of the most beautiful places I’ve ever visited in my life: Machu Picchu. Machu Picchu is of course arguably the most important touristic attraction of Peru.

Machu Picchu

This site is absolutely incredible. I don’t think you can truly capture in images what this place looks like, but I’ve tried to collect a few good ones:

Obligatory selfie Macchu Pichu from the side View from Macchu Pichu Macchu Pichu with Huayna Pichu in the bac

While it’s hard to top Macchu Pichu, we did see a few more things in Peru. We toured around the Sacred Valley for a day and saw some other truly marvelous sites:

I got out of the car during our trip to make this Mara Salt Mines Moray View from Pisac (and my lovely daughter)

In all, Peru was absolutely fantastic. The wedding we attended was very lovely and in an absolutely fantastic spot as well, making a very nice end to our trip.

(Why) I’m stepping down from my WordPress marketing role

joost@joost.blog (Joost de Valk) — Wed, 05 Jun 2019 00:00:00 GMT

I’m going to step away from my role as Marketing Lead. I consider this mostly a personal failure, both in correctly setting and getting expectations and in fitting into another type of organization. Matt and I have talked this through and there are no hard feelings on either side whatsoever. At the same time I’m sad about not having been able to leave more of a mark. Let me explain why I’m stepping down.

When I first talked to Matt about this role he asked me to become “the CMO of WordPress”. In my eyes, a CMO is involved in all aspects of a project / company. When I was announced, I was announced as a “change in WordPress leadership”. My experience over the last few months made me feel that while I was doing things and getting things done, I certainly wasn’t leadership. Which is why I want to step away from my role: I don’t want to pretend I have a say in things I don’t have a say in.

What is marketing?

It seems the problem of defining of what marketing is beforehand, is one of the problems of why I failed in my role. Marketing to me is not just the last step of “promotion”, but the entire process of bringing a product to market. It’s clear that others within the WordPress project don’t necessarily see it like that, and when they describe marketing, it’s a lot more like what I would call advertising. A lot more tactical. I don’t dislike that tactical work, but I think my qualities lie elsewhere.

There’s a stark difference between where I thought I would be in the organization in this role, and where I am actually finding myself now. Even things that every outsider would consider marketing (release posts, about pages) are created without even so much as talking to me or others in the marketing team. Because I felt left out of all these decisions, I feel I can’t be a marketing lead.

Clarity about my position

My position is unclear, not just to me, but to many people which makes me uncomfortable. I’ve been asked dozens of times on Twitter, Facebook and at WordCamps why I now work for Automattic, which of course I don’t but that is the perception for a lot of people. On other occasions I seem to be the token non-Automattician, which I’m also uncomfortable with.

At the same time I feel hampered by my WordPress position to do the work I need to do at Yoast. I notice I’m sometimes shutting up about things Yoast does because that would look weird on the outside and could be perceived wrong. I also felt I was “defending” WordPress too much, on stuff I had otherwise perhaps been more critical of.

WordPress mission and vision

I am used to having a strong vision and mission for a company and a product, and to be translating that into product & marketing decisions. Matt has certainly shown some of his product vision in State of the Words over the year but I’ve found it very hard to get more of the vision behind all the recent changes and the roadmap “out there”.

I’ve not encountered (or been brought into) any discussions about our product vision, something I would need to translate into day-to-day actions. I was expecting there to be some backchannels where these discussions were had and these decisions were made, turns out these simply don’t exist. Matt takes his input from core devchats and lots of other chats and then decides what the roadmap should look like. I honestly think that process needs opening up, even though I do appreciate that Matt has so far been pretty good at bringing the product forward.

An inevitable conclusion

Combined, this doesn’t work for me. I was expecting to be actively involved in larger product and marketing decisions. That didn’t happen. At the same time I have to explain what we do to the outside world and to other people within the WordPress ecosystem, because they assume I know and I’ve been involved. I’m unwilling and unable to do that.

I think some of these things need to change. I see the value in what Josepha is doing and also in projects like the governance project, but these processes take time and patience, and patience is a virtue I’ve not developed well. This was my way of trying to broaden WordPress leadership. I’m sad to conclude that I failed. I’m of course still available to advise and strategize should the project want for that, and will give my opinion, whether I’m asked to or not 😉

Turns out failing burns me out faster than going fast does… That’s why I’m taking an extended holiday this summer, after which I’ll focus my work time for 100% on Yoast and my Chief Product Officer role there. In that role, and outside of it, I’ll absolutely still be an active voice and contributor in the WordPress community.

Update December 2024: A lot has happened in the WordPress community and certainly also the marketing team over the years, prompting me to write about it once more.

Photo by Ugne Vasyliute on Unsplash

Marketing WordPress – first steps

joost@joost.blog (Joost de Valk) — Fri, 22 Feb 2019 00:00:00 GMT

I’ve had the question a few times now of what I’ve been up to since I got appointed lead marketing for WordPress. The first few weeks have been busy, but very interesting. The marketing team is doing a lot of things. I won’t go over all of them here, but wanted to show some of the projects we’ve worked on and how the processes around them work. As you’ll see, most of my focus so far has been on WordPress.org itself and team processes.

Quick wins

During the first weeks one of the things I started working on were some quick wins: making some of our about pages more reflective of the current state of the project. We:

updated the roadmap page (actually a couple of times already, most recently after the 5.1 release);
removed mentions of jQuery and other libraries from the features page and added a lot of links to the support pages to it;
fixed the testimonials page as it was broken;
and finally: started working on a better history page.

Showcase

One of the things that immediately became clear as I was talking to the marketing team is that the showcase needs an update. The marketing team has been publishing great WordPress case studies on the make marketing blog, but those deserve a better place. At the same time the Showcase, where this content would make a lot of sense, obviously needs more love.

When I proposed changing the showcase I got an immediate enthusiastic response from quite a few people, which was very warming. Pragmatic offered to help with design time and has been hard at work based on my (admittedly very simple) outline, leading to some very cool first designs. In the last meeting of the WordPress design team we’ve been discussing their first designs and based on the feedback that came from that, we’ll iterate on those designs a bit more.

This process has already showed me what an enormous ecosystem we have. Let me show you: in this case the marketing team comes up with a plan. The design team, at our request, starts thinking about a design, someone steps up and starts designing, on which the entire design team gives feedback, along with marketing and the accessibility team. Once we’ve agreed on a design, the meta team will start implementing the changes, after which I’m guessing the accessibility and marketing team will once again review the result. At the same time other members of the marketing team are coming up with a process for what a showcase entry will now require and how to get all that data. WordPress really is a huge organization.

Team processes

We’ve made some small adjustments to the team’s processes and I’m certain more will follow. The team’s Trello board is very useful for smaller tasks, but for bigger tasks like the Showcase redesign, we really need to figure out how we can do that more efficiently together with other teams. This is something that I’m hoping (and honestly, expecting) Josepha Haden, the new executive director for WordPress, will help us improve on.

SEO

Last year, as one of the actions leading from the growth council I was part of, Jono Alderson and I did an SEO analysis of WordPress.org (Jono really did the bulk of the initial analysis). Over the last few months we’ve been creating meta tickets to fix those issues one by one. The meta team (the team that handles all things WordPress.org) has been hard at work fixing these, leading to 55 closed tickets at the current count and 19 open ones.

Google Search Console is showing us that errors are going down. It’s also showing a small uptick in traffic that could probably be attributed to these improvements. I expect this is a sign of things to come, as I truly think we can get more traffic to WordPress.org.

Analytics

As we’re working on SEO, Jono and I are also slowly improving the Google Analytics implementation. Using Google Tag Manager we’ve started improving our measurement, for instance by tagging more events. This means we can better track events like downloads, thread creation and replies on the forums and other events.

We have already had the first few requests for analytics data from the Docs and Support teams, who want to use that data to improve their pages. It’s been very nice to quickly be able to answer those as our data gets better.

Is that all?

No, not even close. There is some very exciting work being done by the team on social media campaigns for WordCamps, something I hope we can build and expand on in the future. Another project is the promotion of sustainability for WordCamps. yet another one is the ongoing promotion of WordPress.tv stuff (which probably also deserves a more prominent spot on WordPress.org). There’s also work being done on the mobile page on .org, we’ve started discussing what should happen to the Gutenberg page, and I’m probably still forgetting tons of things as there is so much.

What’s next?

A lot of the above is ongoing work that we’ll have to work on for at least a few more months. At the same time, as I now understand our challenges a bit better, I want to take a step back and produce a “meta view” of how I think we should be marketing WordPress. This marketing plan will then serve as a discussion starter for the marketing team, but I certainly hope also across a wider part of the community.

It’s been very exciting to see what people in the marketing team are capable of. I hope to be able to shine more light on what they do and also to amplify the reach of what they do in the coming months. Combined with my own experience, I look forward to making a meaningful difference in WordPress’ growth.

Update January 2025: my period as marketing lead of WordPress ended rather quickly. I’ve since done many different things around WordPress, most recently, I’ve shared my thoughts around (the current) WordPress leadership.

Stepping “down” at Yoast

joost@joost.blog (Joost de Valk) — Thu, 24 Jan 2019 00:00:00 GMT

Today we’ve announced that Marieke is the new CEO at Yoast, and I’m stepping down. With me recently stepping up as Marketing & Communications Lead for WordPress, this made a lot of sense. In many ways, Marieke was already the acting CEO at Yoast. While this step makes a lot of sense to us, it probably needs some explaining to the outside world.

We do things differently than many other companies do. We run Yoast with the four of us: Marieke, Omar, Michiel and myself. Marieke is always the one that forces us to move forward, to take decisions. Yoast Academy? Her work. Our HR strategy? Her work. Our project management team? Her work. Our research team? Led by her. And that’s not even close to a complete list. Of course there is stuff I do, Omar does and Michiel does, but Marieke is certainly the most prolific of the four of us.

While people seem to think I have been successful it really is we that have been successful. So henceforth I’ll be focusing more on what I am good at, and become Chief Product Officer. I’ll happily say “I don’t know, you’re the boss” from time to time, and that’ll be it. Not all that much will change for me, other than that we’ve switched offices, because she deserves to have the biggest office. But perceptions will change. And that’s good.

You see, all too often people have assumed that Marieke is my secretary. Or that she has a job at Yoast solely because she’s my wife. While she is my wife and I love her dearly, saying that does her great injustice. She’s one of the most capable persons I’ve ever met and I’m happy to be working with her every single day. I’m sure you’ll enjoy seeing her thrive in her new role as much as I do.

Update January 2025: We ended up selling Yoast in August of 2021. I left relatively quickly to join Newfold Digital, the company that acquired Yoast, for another 9 months, after which I left the company to focus on Emilia Capital.

Leading marketing & communication for WordPress

joost@joost.blog (Joost de Valk) — Mon, 21 Jan 2019 00:00:00 GMT

Update: Marieke and I later wrote about the people behind the platform and the role community marketing played in WordPress's growth.

Last week I was appointed Marketing & Communications Lead of WordPress. I think WordPress is one of the most essential platforms on the web and I’m really proud to be able to do my part for it. I have been in the WordPress community for well over a decade now. In that time I’ve done a lot of different things but I think this will be my biggest challenge yet.

WordPress currently powers 32.9% of websites on the web, which, especially in absolute number of sites, is an incomprehensible number. It is also a wonderful project with a great community around it. Many people outside of WordPress don’t know much about the community, something I think we should change. Very often, people don’t even know about the great steps WordPress is making in re-defining content editing. We have to get that knowledge out there. The next phases of Gutenberg will even redefine website building. I hope to be instrumental in spreading this message. Let’s show the 67,1% that’s not using WordPress that maybe they should be.

With the announcement this week has already been tumultuous, but let me describe my next steps below. I’ve also got a couple of questions people have asked a lot, so I’m adding answers to those below.

Next steps

There a few things I’ve already done: I’ve spoken to a lot of members of the marketing team. I had already seen a lot of the things they’re busy with. As I was talking to them I was surprised to find some really high quality work that hardly anybody knew about, that just need a small push. I’m very happy to be able to provide that push. I’m dividing stuff I encounter into two different buckets right now: quick wins and longer term.

I’ve started writing a marketing and communications strategy. This will be based on my own ideas plus all the input I’ve had from people so far. From that strategy we should be able to extract campaigns. Within those campaigns there will be individual tasks that people can pick up.

At the same time I’m looking to update some of the core pages on WordPress.org. These pages describe who we are and what we do. For example: we’ve already updated the Roadmap page, it now reflects our plans for 2019 much better. The Testimonials page that was broken has been fixed, and the Marketing team has a new task to update the History page.

Frequently asked questions

Let me answer the questions I’ve had a couple of times so far:

What does this mean for your day to day?

Obviously this comes with a bit of work. Several people asked me how this combines with my role at Yoast and at home. It’s clear: this is going to have to come out of “Yoast time”. I’m simply not willing to spend less time with my family. Now, I’ll admit: I do work a slight bit more than the average Yoast employee. But saying that, this is still going to impact what I do at Yoast. Most importantly it will probably impact my ability to code as much myself.

What did you think about the discussion around your appointment?

There was some discussion around my appointment, due to the fact that Bridget, the marketing team’s leading representative, had not been informed. This was, oh the irony, a communications issue between Matt, Josepha and myself. We’ve all apologized to Bridget, and I think we’re good now. Of course, I hope I’ll be able to prevent these kinds of communication issues in the future.

Let’s be clear: I’m not replacing her. She was a rep for the marketing team (she has stepped down since) and had done a good job as such. I’ve been given a wider task. I want us to do much more than just write content.

What if there’s a conflict of interest between Yoast and WordPress?

These do come up. And let’s be fair: I’m not exactly the first in WordPress’ leadership who might have a conflict of interest sometimes. Not so long ago I urged people to wait with upgrading to WordPress 5.0. (Btw: you’re fine to update now). While I hope that my deeper involvement in the project will allow me to avoid these situations, when they happen, they happen. I’ll deal with them as transparently as I can. It could mean that I, with my WordPress hat on, have a different opinion than Yoast, the company.

Are you being paid for this?

No. This is a volunteer position. I do benefit indirectly from WordPress’ growth, as that’ll probably mean Yoast will grow too, but I don’t think you can really call that payment. The fact that my position is unpaid does not mean there won’t be any paid positions within my team. I think marketing for WordPress as a whole would definitely benefit from having a few people that are able to consistently spend more than a few hours a week on it.

Are you going to fix the ambiguity between WordPress.com and WordPress.org?

No. There are limits to what I’m allowed to change and this falls outside of those limits. Not that I’d even want to: I don’t feel like it’s that much of a problem, it’s definitely an ambiguity but both sides benefit from it too.

I do think that we’re not always doing a good job of making the distinction between the two. Just pointing it out to f.i. journalists when they make the mistake might clear some things up and make the distinction more widely understood.

Does this mean Automattic is acquiring you?

It’s surprising how many people have asked me this or similar questions in the past few days. The answer is simple: No. Automattic is not acquiring us. The reason for that is simple: we’re not for sale. We’re having fun, and we’re not looking to be acquired.

Update January 2025

My period as marketing and communications lead for WordPress ended rather quickly. I’ve since done many things around the WordPress project, and invested in many companies in the WordPress space. Most recently I’ve shared my thoughts on (the current) WordPress leadership.

Karen Sparck Jones

joost@joost.blog (Joost de Valk) — Thu, 03 Jan 2019 00:00:00 GMT

“Computing is too important to be left to men.”

I had never heard of Karen Sparck Jones before, I’m thankful I have now:

https://www.nytimes.com/2019/01/02/obituaries/karen-sparck-jones-overlooked.html

We have a team of linguists / developers at Yoast, looks like we owe her a great deal.

Democratizing publishing

joost@joost.blog (Joost de Valk) — Thu, 27 Dec 2018 00:00:00 GMT

Update: Six years later, the question of what "democratizing publishing" means in practice became central to the WordPress governance discussion.

Matt blogged, in response to a question asked at WordCamp US: what does “democratizing publishing” mean to you? His answer:

… the mission of “Democratize Publishing” to me means that people of all backgrounds, interests, and abilities should be able to access Free-as-in-speech software that empowers them to express themselves on the open web and to own their content. …

Matt Mullenweg in his blog post

I agree with this explanation of the mission, it’s something we at Yoast deeply believe in. We also want to further it with Yoast SEO by making it possible for everyone to not just express themselves, but also to be found. If people want to express themselves, a large chunk of those people also want those expressions to be found by others. While search engines aim to get everything findable, getting ranked so other people see you takes a bit more work. Hence our own mission: SEO for everyone.

After WCUS 2018: WordPress & Gutenberg FAQ

joost@joost.blog (Joost de Valk) — Wed, 12 Dec 2018 00:00:00 GMT

I just came back from WordCamp US in Nashville, which was awesome. In this post I want to address some of the questions I’ve had a couple of times. So here we go:

**What is it you don’t like about Gutenberg?**There is nothing I don’t like about Gutenberg. Ok, that might be a bit of an exaggeration but in general, I seriously love it. Gutenberg makes WordPress ready for the web of today. That’s the reason our development team spent so much time to help make Gutenberg better.

**How well does Yoast SEO do with Gutenberg?**It’s fine! In fact, it’s better than ever. Gutenberg has some performance issues (in the admin) at the moment that are more obvious when you use Yoast SEO, but I expect the next few minor releases of WordPress 5.0 to fix those. If you’re suffering from them now, installing and activating the Gutenberg plugin and keeping that up to date will get you those changes earlier.

**But I thought you were mad about the release?**I was annoyed with the communication failure around the release. Initially, WordPress 5.0 was slated for November. In the post outlining that, a specific choice had been made to skip December, should November not be feasible, and go to January. Then, when the November timeframe proved unfeasible, the team tracked back on that and Matt decided to release in December anyway, with 2 days notice, despite several people’s objections, including my own.

I still do not agree with the timing of the release, but it’s done now. Matt has offered his apologies for the poor communication around that and with that, we’re now looking onward. I’m looking forward to seeing how we, as a WordPress community, can improve the consistency of our communication in the future.

**So should I upgrade to WordPress 5.0 now?**Our suggestion is still to wait, as I said in my post on Yoast.com on WordPress 5.0. Yoast SEO is awesome with Gutenberg and I can’t wait for everyone to try it, but if you’re going to try it now, you might think that it’s annoying and slow. If you use it with Gutenberg 4.7 (currently in RC phase, you can get it here), it’s already much much better. This means that if you update in January, you should certainly be OK.

**Is your support team suffering under WordPress 5.0?**Honestly: no. It seems to be doing reasonably fine, with very limited support on our end. Only one plugins so far is left that is known to break Yoast SEO’s meta box: Gravity Forms’ Gutenberg Add-On (which is still in beta). I’m assuming that will get fixed.

Two plugins have already been updated:
– WPML you should update to 4.1.2 or higher.
– Pods has been fixed in 2.7.11, so update to that, or a newer version.

In general, many plugins will probably need to update a few more times the next few days and weeks.

**Should people stick with the classic editor?**No! You’re missing out on lots and lots of cool stuff, so sticking with the classic editor should not be a long term solution. It might be fine for a few months as plugin and theme developers work out their issues with Gutenberg, but I’d suggest switching it on as soon as you can.

There are always going to be people who don’t like interface changes, and such is the case with this release too. The awesome teams in the WordPress forums are helping those people where they can and I assume a few of them will continue using the classic editor for now.

**What are you looking forward to in WordPress?**In Matt Mullenweg’s State of the Word he announced a longer term roadmap than we’ve had for a while, something I really like. He announced the next phase of Gutenberg, which looks promising, and 9 focus points for the next year, which all seem to be very nice projects. On top of that he announced the intent for WordPress core to support multi-lingual sites, something I’ve been asking for as a core feature for quite some time now.

**Was this post just an excuse to use the Yoast SEO FAQ block?**No, not really… Well, maybe a little 😉

My experience: I had to switch to using the Gutenberg 4.7 RC as I was doing this. Otherwise it does indeed become slow when you have more than ~ 5 questions in an FAQ block. As soon I switched to the 4.7 RC though, it immediately became useful and in fact, very nice to work with.

At the same time, I’ve found some minor issues that we’re going to have to solve in our FAQ blocks, so those will get an update too.

Have more questions for me? Drop them in the comments, I’ll answer them there!

WordPress 5.0 needs a different timeline

joost@joost.blog (Joost de Valk) — Tue, 06 Nov 2018 00:00:00 GMT

For the last few months, the WordPress developer community has been moving towards a release of WordPress 5.0. This is the highly anticipated release that will contain the new Gutenberg editing experience. It’s arguably one of the biggest leaps forward in WordPress’ editing experience and its developer experience in this decade. It’s also not done yet, and if we keep striving for its planned November 19th release date, we are setting ourselves up for failure.

Update November 9th: WordPress 5.0 has been moved
The new release date is November 27th, 2018. See the Make/Core post for details. While I’m happy that it’s been postponed, I’m not sure whether this is enough to get WordPress 5.0 to be as accessible and stable as I think it should be. Time will tell, I guess.

Let me begin by stating that I love Gutenberg. It’s the best thing since sliced bread as far as content editing is concerned. I’m writing this post in Gutenberg. I started writing it on my iPhone. It rocks. But it also still has numerous bugs. In fact, the editor broke on me during writing this post and failed to autosave all the contents. Luckily I saw it breaking and copied the paragraphs to an external editor.

Reasons for delaying

There are a two main reasons why the November 19th timeline is in my opinion untenable:

There are some severe accessibility concerns. While these aren’t new and a few people are working hard on them, I actually think we can get a better handle on fixing them if we push the release back. Right now it looks to me as though keyboard accessibility has regressed in the last few releases of Gutenberg.
The most important reason: the overall stability of the project isn’t where it needs to be yet. There are so many open issues for the 5.0 milestone that even fixing all the blockers before we’d get to Release Candidate stage next week is going to prove impossible. We have, at time of writing 212 untriaged bugs and 165 issues on the WordPress 5.0 milestone.

People are working hard

The amount of work being done every day right now by the development team is bordering on the insane. Look at the work for the last three days:

I’d normally be happy with this for a week. This is 3 days, also including a Sunday. It’s been like this for a while. I appreciate all these people doing the hard work, but moving this fast only increases the chance of regressions.

When I mentioned earlier today in the WordPress Slack’s #core-editor channel that I think we should push back, the response was pretty positive:

Let’s get this straight: this is in the channel with a large part of the people working on this release. I’m not the first to say this. I hope this post will help the powers that be come to the same conclusion.

Conclusion: push back, and zoom out

All these things considering, my conclusion is simple: we need to push back the release. My preference would be to January. This would allow us to zoom out a bit, prevent regressions and overall, lead to a better product, with finished documentation. Something that’s worthy of the label RC when we decide to stick that on it. Right now, I feel that the beta is more of an alpha, and we’ll end up with an RC that’s more of a beta.

“Poetry is not an expression of the party line. It’s that time of night, lying in bed, thinking what you really think, making the private world public, that’s what the poet does.”

Allen Ginsberg, from Ginsberg, a biography

Gutenberg and Yoast SEO

joost@joost.blog (Joost de Valk) — Sat, 17 Jun 2017 00:00:00 GMT

I’m typing this as I sit in Track 1 of WordCamp Europe in Paris. Matt Mullenweg just announced Gutenberg as a plugin is available, so I installed it on here and am writing this post in it.

This was the beginning of an interesting number of posts, eventually asking for a different timeline for Gutenberg. In many ways, this was the beginning of a lot of discussion about leadership.

Gutenberg is a radical rethinking of the post screen and that means we at Yoast have to rethink how we integrate with it. Some of what we do, our real time content and SEO analysis will still have to be visible all the time. Our snippet preview though, might actually move to a pre-publish workflow, a concept that the Gutenberg team has been thinking about.

None of this is set in stone yet, I look forward to building an integration that works well. I would also love, if you’re reading this, your ideas on how we could integrate with this.

Accessibility and Gutenberg

One of the things that we at Yoast worry about is the accessibility of Gutenberg. We have a team member, Andrea Fercia, who’s very active as a WordPress contributor. We have freed him up to work on accessibility on this project so it can be merged into WordPress soon without losing the accessibility that the current editor has.

Install Gutenberg

You should really install Gutenberg on a test site and play with it: Install Gutenberg.

Blogging about not SEO

joost@joost.blog (Joost de Valk) — Sat, 14 Jan 2017 00:00:00 GMT

I’ve been wanting to do this for a while: blogging for fun, on a domain separate from Yoast.com. Blogging about stuff I do outside of Yoast, which, surprisingly to some people, is more and more. I will blog here about the investments Marieke and I make with Altha, our investment company and my advisory work, when I’m allowed to.

I’ll also use this to vent my political opinions every so often. I like discussion, so, if you do too, I’ll see you in the comments 🙂

Update January 2025: This post was written in 2017, 8 years later, this is still my blog. The only change: Altha was renamed to Emilia Capital a few years ago and became a bit more important when we sold Yoast.