See also:

• Brief view of the dispatcher and its goals
• Initial settings for enable cache

Many of us might have met the situation when the dispatcher uses an old code version. This article describes how to avoid this while leveraging dispatcher caching possibilities.

Invalidation is a mechanism for identifying obsolete cached resources. There are some tools for automatic invalidation and manual invalidation. Let’s set the initial configuration for the invalidation section of the dispatcher configuration file, then learn how invalidation works at the low level, and finally, return to analyze invalidation tools. But basics first.

What is Dispatcher cache in AEM?

The Dispatcher is AEM caching and load-balancing tool that stores cached files on the web server the same way a static website does. Once requested, a cacheable document is checked by the Dispatcher to identify whether that document exists in the web server’s file system:

• If the document is cached, the Dispatcher returns the file.
• The Dispatcher requests the document from the AEM instance if it is not cached.

Serving the data from the Dispatcher’s cache decreases the load on AEM publish instances.

/cache
{
    /invalidate
    {
        /0000  { /glob "*" /type "deny" }
        /0001  { /glob "*.html" /type "allow" }
    }
}

How to clear Dispatcher cache?

There are at least 3 ways to clear the Dispatcher cache:

• By flushing agents on Author and/or on Publishes
• Manual flush
• Custom code to do the flush through programming

Invalidation section initial settings

Let’s now get to invalidating cached pages from AEM. The /invalidate block inside the /cache section determines cached files that may be automatically invalidated when content is updated. For example, the following configuration invalidates all HTML pages:

/cache
{
    /invalidate
    {
        /0000  { /glob "*" /type "allow" }
    }
}

With automatic invalidation, the dispatcher doesn’t delete cached files after updating content but checks their validity when they are next requested. Documents in the cache that are not auto-invalidated will remain in the cache until they are deleted during content updates. To cite an example, let’s allow all the cache to be invalidated automatically:

Restart the httpd server after updating /invalidate section for using new changes.

Invalidation in depth

The Dispatcher uses special empty files named by default “.stat” at the low level. The /statfileslevel “0” parameter is set by default, which means that only one stat-file located in the root of the htdocs folder is used. If the time of changing the stat-file is later than the time of changing the resource, the dispatcher considers such a resource obsolete or invalid.

For example, we have these cached resources after requesting the page http://localhost/content/geometrixx/en/products.html :

Let’s invalidate them by applying a low-level stat-file mechanism. Create an empty file named “.stat” in the root of your htdocs directory:

You may see that the time of the statfile change is later than the time of the cached resources. For the dispatcher, this means that all resources are out-of-date. This is a low-level invalidation mechanism. If we visit http://localhost/content/geometrixx/en/products.html again after creating this stat-file, the requested cached resources will be updated:

This example illustrates an invalidation pattern with the default /statfileslevel parameter “0”. Let’s see how we can fine-tune the invalidation in more detail with the /statfileslevel parameter.

Setting /statfileslevel

You may use the /statfileslevel properties of the dispatcher configuration file to selectively invalidate cached files according to their path. There are some rules for /statfileslevel properties mechanism:

• The dispatcher creates .stat files in every folder, starting from docroot and up to the level you specify. The level of the docroot folder is 0.
• When you update a file, the dispatcher finds a folder located at the statfileslevel level and invalidates all files in that folder and all files that lie below within that folder.
• If the level of the updated file is lower than the statfileslevel, then only the files of the folder containing the updated file are disabled, but the files lying below within this folder remain valid.
• When updating a file, all files of the corresponding folder and above up to and including the root level will become invalid.

Let’s look at a couple of examples to better understand the rules of the /statfileslevel property. Our default demo case, where /statfileslevel is “0”, looks like this:

There is only one stat-file in the docroot root folder. And the area of responsibility or scope of this stat-file will be the whole file tree of our docroot. If some file in the tree has a modification date older than the modification date of the stat-file, the dispatcher will consider this file invalid.

If we set /statfileslevel “4”, the invalidation works like this:

Stat files exist within all levels from 0 (root) to 4.

A stat-file at a level less than 4 has a scope or area of responsibility only for the folder containing it. If a stat file inside the content/geometrixx/en folder is newer than any file from this folder, then this file is invalid, but all files from other folders are defined by other stat-files.

Only stat files with the value statfileslevel (in our case, the level is 4) have the scope or area of responsibility of the entire underlying tree, starting from the folder containing such a statfile and extending down to the lower levels of the file tree.

So If a state file inside the content/geometrixx/en/products folder has a newer modification time than any file in the underlying file tree, which includes the products folder, the dispatcher considers this file invalid. Validation of all files which are not in this tree is determined by other stat files.

Automatic invalidation and Dispatcher flush agent AEM

To automate invalidation, you can enable authoring or publish flush-agents. We recommend using the publish flush agent for more reliable automatic invalidation, as using the authoring flush agent can cause the following issues:

• The dispatcher must be accessible to the author’s AEM server. If your network (e.g., because of a firewall) is set up so that access between both is denied, automatic invalidation won’t work.
• Publishing and cache invalidation occur simultaneously. A user can request a page immediately after it is removed from the cache but before the page has been published. In this situation, AEM returns the old page, and the dispatcher caches it again and considers it valid now. This problem affects large sites primarily.

The public flush agent is located at: http://localhost:4503/etc/replication/agents.publish/flush.html

To enable your publish flush agent, click the “Edit” button, and set the “Enabled” checkbox:

Update the URI port on the Transport tab and set its value to 80:

Save your updates, and you will see that publish flush agent is activated:

Manual invalidation requests

You can send the following requests to manually invalidate your cached resources:

• to delete cached files

POST /dispatcher/invalidate.cache HTTP/1.1
CQ-Action: Activate
CQ-Handle: path-pattern
Content-Length: 0

• to delete and recache files

POST /dispatcher/invalidate.cache HTTP/1.1
CQ-Action: Activate 
Content-Type: text/plain
CQ-Handle: path-pattern
Content-Length: numchars in bodypage_path0
Page_path1
…
Page_pathn

Summary

As a result, we managed to find out how these things work:

• Low-level invalidation mechanism;
• Flush agents for automatic invalidation after pages are published;
• Manual invalidation queries.

More detailed and useful documentation is available on the following pages:

http://docs.adobe.com/docs/en/dispatcher/disp-config.html

http://docs.adobe.com/docs/en/dispatcher/page-invalidate.html

FAQ

How to clear dispatcher cache in AEM?

There are at least 3 ways to clear the Dispatcher cache in AEM:

• By flushing agents on Author and/or on Publishes
• Manual flush
• Custom code to do the flush through programming

What is dispatcher cache in AEM?

The Dispatcher is AEM caching and load-balancing tool that stores cached files on the web server the same way a static website does.

How to check dispatcher logs in AEM?

The dispatcher logs are controlled within the dispatcher module configuration. Based on your setup, this can either be a separate .conf file in /etc/httpd/conf.d or within the same file as the Dispatcher vhost configuration.

What is the difference between cache flush vs invalidate in AEM?

According to the documentation, both flush and invalidation mean the same.

“To invalidate (or flush) the Dispatcher cache without activating a page, you can issue an HTTP request to the dispatcher.”

But it’s safe to say that invalidation is an HTTP call to the dispatcher to mark the cached resource as invalid (the same occurs when the resource’s TTL expires). While flush usually means invalidation triggered from the Author/Publish instance when the content is published.