Thursday, March 19, 2020

The Annotated URL (aURL)

Here is a modest proposal for a URL annotation scheme that is useful when downloading static resources via URLs and offers a robust alternative to HTTP Cache-Control.

We have used this system for many years to reliably transfer terrabytes of data to hundreds of thousands of devices around the world. It could also be integrated into web browsers to significantly reduce the number of HTTP requests.

Specification

Annotated URL (aURL) parameters let you check cached files without making a new web request:

    <url>?date=<unixtime>&size=<bytes>&md5=<hash>

For example:

    https://www.google.com/favicon.ico?date=1481158857&size=5430&md5=f3418a443e7d841097c714d69ec4bcb8

Where:

  • &date lets you check if the remote file is newer than the local cached file and needs to be downloaded again.
  • &size gives a trivial check of file integrity without needing to hash the file and tells you the size before the request begins.
  • &md5 is between 1 and 32 hex nybbles of the MD5 file hash where increasing character count gives increasing confidence of integrity.

All aURL parameters are optional, but all three are recommended because URL characters are cheap and web requests are expensive.

aURLs can also be used to verify the integrity of a newly downloaded file with no additional information.

aURLs are CDN friendly because they naturally force the edge nodes to fetch modified resources.

aURLs could be generated automatically by publishing platforms like WordPress.


No comments: