Attempting to work around the 10 minute delay

[Lee_Ars]

I’m planning on using the wp-discourse plugin on a gulf coast weather forecasting site that typically services ~10-20k pageviews/day, but occasionally during severe weather events will push 1.5-2MM PV/day to approx 500-700k visitors. The site + its hosting strategy have been battle-tested through several severe weather events and things work great under pressure, primarily thanks to careful design and a lot of help from Cloudflare.

The site’s users are used to a low-friction commenting experience via native Wordpress comments, so there will be some adjustment (like getting them used tgo clicking the “Continue the discussion at…” link) that the user-facing staff is ready to manage.

However, what they won’t tolerate is the variable 10-minute delay between posting comments and them being shown on the daily Wordpress post’s page. They will want new comments (up to the configured limit) immediately on the homepage upon posting, similar to how native WP comments are displayed immediately after posting.

After screwing around with the built-in options to try to make posts show up immediately without nginx’s fastcgi caching, wordpress, or browser caching interfering with new comments appearing on refresh after they’re posted, the I have added the following two mu-plugins to mitigate this and cause newly posted comments to show up on the Wordpress side on refresh:

wp-discourse-transient-killer.php
wp-discourse-cache-header-fix.php

This has solved my problem: new posts to Wordpress-created Discourse threads now appear below the wordpress posts instantly on refresh.

But I’m kind of at the edge of my competence here — what am I breaking/screwing up/undermining by doing this?

I don’t particularly care about creating additional load on my webhost by having the comment endpoint being hammered by visitors checking the daily weather forecast page (with its embedded Discourse comments) — that is a problem I can solve by throwing money at it. My primary requirement is avoiding 20,000+ users sending me emails asking me why their comments aren’t appearing instantly on the homepage when they post them.

Is this the right approach? Is what I’m doing wise? Does it create additional security or performance issues I haven’t anticipated? Basically, am I screwing things up by doing this?

Thanks

[RGJ (Richard - Communiteq)]

Hmm I’m confused why this works, because

@angus

The action will receive the ids of the Wordpress posts that have been updated by the webhook as an argument, i.e.

do_action( 'wpdc_after_webhook_post_update', $post_ids );

And your plugin

add_action( 'wpdc_after_webhook_post_update', function( $topic_ids ) {
    foreach ( (array)$topic_ids as $topic_id ) {
        delete_transient( 'wpdc_comment_html_' . $topic_id );
    }
}, 11 );

Aren’t you mixing up the (Wordpress) $post_ids passed to the action with the (Discourse) topic_id that serves as the transient key ?

I would expect your plugin to look like this

add_action( 'wpdc_after_webhook_post_update', function( $post_ids) {
    foreach ( (array)$post_ids as $post_id ) {
        $topic_id = get_post_meta( $post_id, 'discourse_topic_id', true );
        delete_transient( 'wpdc_comment_html_' . $topic_id );
    }
}, 11 );

OTOH you’re saying this works?

[angus (Angus McLeod)]

Hey @Lee_Ars, can you first confirm that you have the comments webhook setup?

Configure WP Discourse Webhooks

You can use webhooks to sync data between Discourse and WordPress, and enrich your integration. Reply to this topic if you have questions about how to setup the webhooks, or how they work. Setup WP Discourse First setup an endpoint in WP Discourse: Navigate to WP Discourse > Webhooks in Wordpress admin. Select the endpoint(s) you want to enable (see below). Set a “Webhook Secret Key”, if it’s not already set. Click “Save Options”. Discourse Then create a new Webhook in Discourse (Go to /admi…

That works like so:

1. There’s a new post in Discourse.
2. A webhook payload is sent to Wordpress.
3. WP discourse updates the comment count on the post and also sets a post custom field wpdc_sync_post_comments.
4. If wpdc_sync_post_comments is set Discourse comments will sync when a Wordpress post is loaded, regardless of the sync period (i.e. the 10 minute delay).

Before we get into caching, I just want to be sure that is in place first. If it is, perhaps also just turn on “Verbose Webhook Logs” and verify you’re receiving webhook requests when a new post is made in Discourse.

[Lee_Ars]

@angus

Hey @Lee_Ars, can you first confirm that you have the comments webhook setup?

Hi Angus! That’s affirmative, the “Sync Comment Data” webhook option is enabled on the WP side, and I created the webhook on the Discourse side and pings are successful. The WP plugin’s logging shows comment.INFO: sync_comments.success messages at the right timestamps:

[2025-07-07 14:16:38] connection.INFO: check_connection_status.successful_connection  
[2025-07-07 14:16:38] connection.INFO: check_connection_status.valid_scopes  
[2025-07-07 20:11:31] comment.INFO: sync_comments.success {"post_id":786} 
[2025-07-07 20:25:03] comment.INFO: sync_comments.success {"post_id":786} 
[2025-07-07 20:32:14] comment.INFO: sync_comments.success {"post_id":786} 
[2025-07-07 20:44:15] comment.INFO: sync_comments.success {"post_id":786} 
[2025-07-07 21:00:39] comment.INFO: sync_comments.success {"post_id":786} 
[2025-07-07 21:01:42] comment.INFO: sync_comments.success {"post_id":786} 
[2025-07-07 21:15:40] comment.INFO: sync_comments.success {"post_id":786} 

It’s just that even with those success messages, existing visitors (or at least me, multiply testing on firefox/safari/chrome on a mac, firefox/chrome/edge on a win10 PC, and safari on ios) keep getting a cached /wp-json/wp-discourse/v1/discourse-comments endpoint, with cache-control headers set to a nonzero amount. If I do a ctrl-shift-f5 on Chrome (or the equivalent in other browsers) to force it to bypass local cache on refresh, everything works great and the new posts appear.

With the mu-plugins in place, that endpoint shows cache-control set to no-store no-cache etc, and the problem behavior doesn’t manifest—simply visiting the WP post or refreshing it with the regular refresh button shows new embedded comments.

I’ve turned on verbose webhook logging and posted a test post, and things appear OK when I create a new post:

webhook_topic.INFO: update_topic_content.update_post_metadata_success {"post_ids":"786"}

Everything seems to be working, but I still don’t really understand why the original problem behavior manifested, or if it’s arisen from some overlooked issue on my end. (Very possible!)

@RGJ

Hmm I’m confused why this works…

D’oh, sorry, I’m sure you’re right about me screwing that up — yesterday was a long day, and like I said, I’m kind of at the limit of my abilities here. It was definitely doing something, though now I’m wondering if the “fixed” behavior I’m seeing is just it being broken in a new way. (I’ve updated the “transient killer” plugin to use the right argument now, thank you!)

[angus (Angus McLeod)]

Thanks, so just one more thing to clarify. Is this WP Discourse setting (in “Commenting”) on?

[Lee_Ars]

I’ve tried both with the Cache Comment HTML setting enabled or disabled and it appears to have no impact on the problem behavior. I’ve currently got it disabled enabled (sorry, I was incorrect, it’s currently enabled while I keep poking at it), but I can set it to whatever would be helpful for troubleshooting.

[RGJ (Richard - Communiteq)]

If the setting is disabled, you won’t notice the topic_id / post_id mixup, since that plugin does nothing in that case. No caching → it doesn’t matter if you delete the wrong cache.

If the setting is enabled then you should notice that the plugin doesn’t work properly.

I.e. if you want to troubleshoot, you should enable the setting.

[angus (Angus McLeod)]

Ok, so as a first measure for your case I’d suggest:

1. keep Cache Comment HTML disabled; and
2. remove the https://www.bigdinosaur.org/r/wp-discourse-transient-killer.txt plugin as it’s currently not doing, anything as Richard observes.

If that does not resolve the problem then the problem is to do with another form of caching. The next questions to answer are:

1. What caching solutions do you (and / or your hosting provider) use for Wordpress?
2. If https://www.bigdinosaur.org/r/wp-discourse-cache-header-fix.txt fixes the issue, how does it’s specific behaviour invalidate whatever cache(s) are being applied in 1?

Looking at wp-discourse-cache-header-fix I see that one of the fixes is for load-comments.js. Do you have this setting enabled?

[Lee_Ars]

@angus

What caching solutions do you (and / or your hosting provider) use for Wordpress?

This is self-hosted WP on nginx+php-fpm 8.3 with nginx fast-cgi caching for dynamic content and Redis object caching (with the object cache drop-in active). There are no other layers (no CDN, no CF, no on-box Varnish or other local cache beyond nginx’s fast-cgi cache). Dumping the nginx fast cgi cache (aggressively, by going rm -rf /etc/nginx/cache/*) has no effect on the problem behavior — stale results are served even after nuking the cache directory and restarting both nginx and php-fpm.

I do indeed have the Ajax comment loading enabled right now, yes, but again, toggling it off (and dumping nginx’s cache plus restarting nginx and php-fpm just in case) had no effect on the problem behavior. Browsers still got stale comments.

@angus

Ok, so as a first measure for your case I’d suggest:

1. keep Cache Comment HTML disabled; and
2. remove the https://www.bigdinosaur.org/r/wp-discourse-transient-killer.txt plugin as it’s currently not doing, anything as Richard observes.

Option toggled, transient-killer removed. No change in problem behavior.

@angus

If https://www.bigdinosaur.org/r/wp-discourse-cache-header-fix.txt fixes the issue, how does it’s specific behaviour invalidate whatever cache(s) are being applied in 1?

The effect it applies appears to be supplying a no-cache cache-control header instead of one with a cache time specified. WIthout it in place, my browser seems to very much want to serve a stale cached version of the wp-json/wp-discourse/v1/discourse-comments endpoint from its disk cache; as noted, i have to shift-ctrl-f5 (or the equivalent) to force a no-cache refresh.

The problem behavior seems to be on the browser end, rather than in a persistent server cache. It’s just every browser on every OS that I have access to that’s doing it.

[angus (Angus McLeod)]

Ok, just so I’m 100% clear, when you have:

• verified working comment webhook
• Cache Comment HTML off
• AJAX Loading off
• no cdn
• no cloudfront
• no wordpress caching plugin
• no relevant warnings or errors in the php logs

you’re sure it’s not working?

If it’s not working with that setup, I’m at a bit of a loss without having a closer look and would default to

• AJAX loading on
• Your wp-discourse-cache-header-fix.php fixes

which is what I suspect was working for you. If that route is working, then you should go for that.

[Lee_Ars]

@angus

Ok, just so I’m 100% clear, when you have:

• verified working comment webhook
• Cache Comment HTML off
• AJAX Loading off
• no cdn
• no cloudfront
• no wordpress caching plugin
• no relevant warnings or errors in the php logs

Here’s a quick imgur gallery of screenshots of my current plugin settings, for reference.

Confirm no CDN, no Cloudfront or Cloudflare either one, no caching plugins aside from the Nginx helper (to help WP invalidate nginx fast-cgi cache as needed).

Also confirm nothing relevant in either the php-fpm or the nginx error logs.

you’re sure it’s not working?

God, dude, i wish it were. I’ve been banging my head on this for about 30 hours at this point, with some time off to sleep. I might be going a little crosseyed, heh.

[angus (Angus McLeod)]

Yeah, I feel you. Take a break for a day or so. I’ll try to recreate the issue tomorrow by copying your setup.

[Lee_Ars]

If I don’t flail my way to a solution, and if you’d like, I’d be happy to grant some temporary local access (to the WP blog, the discourse, and/or the underlying hosts) tomorrow or the next day for troubleshooting. Definitely not trying to get free labor or anything, though. Would be happy to pay for your services if there’s actual time to be spent here.

For reference, both the host holding Wordpress and the host running Discord are t3a.large EC2 instances running AWS’ Ubuntu 24.04 AMI; installed software is minimal (nginx + php8 + fpm + redis on the web host, and nothing beyond git + docker + discourse on the discourse host). This is my personal blog, and I’m testing comment integration here before I roll it out in production on the much larger WP site mentioned in the OP. (I also maintain that site and it has a similar tech stack, though it’s fronted by Cloudflare. I’ll cross that bridge when I come to it!)

If you need/want any more details about the underlying stack, I’m happy to provide them.

[angus (Angus McLeod)]

Ok, here’s a video of me trying (and failing) to repro your issue:

Setting Up WordPress With Discourse Discussion Forum Software

The next thing I want you to try is this filter.

@angus

wpdc_comment_sync_period

Set the sync period for comments in seconds. Default 600, i.e. 10 minutes.

Example

Set the sync period to 5 minutes.

function custom_comment_sync_period( $minutes, $post_id ) {
  return 300;
}
add_filter( 'wpdc_comment_sync_period', 'custom_comment_sync_period', 10, 2);

[Lee_Ars]

Thanks @angus — failing repro actually makes me feel a lot better, because that means I’m doing something wrong rather than there actually being something wrong

I will get that filter added today when I have a block of time for troubleshooting and report back