Magick Cache: an Efficient Image Cache
The Magick cache is a work in progress. Do not use the cache in production services until version is at least 1.0.0. It is currently 0.9.2. The Magick Cache requires ImageMagick version 7.1.0-0 or above.
The MagickCache provides methods and tools to cache images, image sequences, video, audio or metadata in a local folder. Any content is memory-mapped for efficient retrieval. Additional efficiences are possible by retrieving a portion of an image. Content can persist or you can assign a time-to-live (TTL) to automatically expire content when the TTL is exceeded. MagickCache supports virtually unlimited content upwards of billions of images making it suitable as a web image service.
Create a Magick Cache
You'll need a place to store and retrieve your content. Let's create a cache on our local filesystem:
$ magick-cache -passkey passkey.txt create /opt/magick-cache
Where `passkey.txt` contains your cache passkey. Don't forget your passkey. Without it, you will be unable to identify, expire, or delete content in your cache.
Once its created, you will want to populate the cache with content that includes images, video, audio, or metadata.
Put content in the Magick Cache
Let's add a movie cast image to our newly created cache:
$ magick-cache put /opt/magick-cache movies/image/mission-impossible/cast/rebecca-ferguson 20210508-rebecca-ferguson.jpg
Note, the image identifier is an IRI composed of project/type/resource-path. In this example, the project is
movies, type is
image, and the resource path is
mission-impossible/cast/rebecca-ferguson. The path uniquely identifies a resource. Two different images cannot be stored with the same resource path. Instead use something like
Now, set a resource passkey and the time to live to 2 days. Anytime after 1 day, the image will automatically expire with the
expire function. To get, expire, or delete the image, you will need to use the same resource passkey.
$ magick-cache -passkey passkey.txt -ttl "2 days" put /opt/magick-cache movies/image/mission-impossible/cast/rebecca-ferguson 20210508-rebecca-ferguson.jpg
passkey.txt contains your resource passkey. Don't forget your passkey. Without it, you will be unable to get, identify, expire or delete your content.
The resource passkey ensures only you and the cache owner can access your image. To prevent the cache owner from viewing its content, scramble it with:
$ magick-cache -passkey passkey.txt -passphrase passphrase.txt -ttl "2 days" put /opt/magick-cache movies/image/mission-impossible/cast/rebecca-ferguson 20210508-rebecca-ferguson.jpg
Note, blobs and metadata are stored in the cache in plaintext. To prevent snooping, scramble its content before you store it in the cache.
Get content from the Magick Cache
Eventually you will want retrieve your content, let's get our cast image from the cache:
$ magick-cache -passkey passkey.txt get /opt/magick-cache movies/image/mission-impossible/cast/rebecca-ferguson rebecca-ferguson.png
Notice the original image was put in the cache in the JPEG format. Here we conveniently convert it to the PNG image format.
-extract option is useful when retrieving an image. To extract a portion of the image, specify tile width, height, and offset:
$ magick-cache -passkey passkey.txt -extract 100x100+0+0 get /opt/magick-cache movies/image/mission-impossible/cast/rebecca-ferguson rebecca-ferguson.png
To resize instead, do not specify the offset:
$ magick-cache -passkey passkey.txt -extract 100x100 get /opt/magick-cache movies/image/mission-impossible/cast/rebecca-ferguson rebecca-ferguson.png
If your image is scrambled, provide the passphrase to descrample it:
$ magick-cache -passkey passkey.txt -passphrase passphrase.txt get /opt/magick-cache movies/image/mission-impossible/cast/rebecca-ferguson rebecca-ferguson.png
Delete content from the Magick Cache
We can explicitly delete content:
$ magick-cache -passkey passkey.txt delete /opt/magick-cache movies/image/mission-impossible/cast/rebecca-ferguson
or we can delete cast images that have expired (exceeded their respective time to live), try this comand:
$ magick-cache -passkey passkey.txt expire /opt/magick-cache movies/image/mission-impossible/cast
Identify the Magick Cache content
Perhaps you want to identify all the content you own:
$ magick-cache -passkey passkey.txt identify /opt/magick-cache movies/image/mission-impossible/cast movies/image/mission-impossible/cast/rebecca-ferguson[1368x912] 406B 1:0:0:0 2021-05-30T17:41:42Z identified 1 resources
Each entry includes the IRI, image dimensions for images, the content extent in bytes, time to live, whether the resource is expired (denoted with a `*`), and the creation date.
Others can store content in the cache along side your content. However, their content is unavailable to you. You cannot get, identify, expire or delete content that you do not own as determined by your secret passkey.
The magick owner can get, identify, expire, or delete all the content, including content you own, with this command, for example:
$ magick-cache -passkey passkey.txt identify /opt/magick-cache /
Note, expired reesources are annotated with an asterisks.
Magick Cache is not just for Images
In addition to a type of
image, you can store the image content in its original form, video, or audio as content type of
blob or metadata with a content type of
$ magick-cache -passkey passkey.txt put /opt/magick-cache movies/blob/mission-impossible/cast/rebecca-ferguson 20210508-rebecca-ferguson.mp4
$ magick-cache -passkey passkey.txt put /opt/magick-cache movies/meta/mission-impossible/cast/rebecca-ferguson 20210508-rebecca-ferguson.txt
Images must be in a format that ImageMagick understands. Metadata must be text. Blobs can be any content including images, video, audio, or binary files.
Delete a Magick Cache
To completely delete all the content within a cache:
$ magick-cache -passkey passkey.txt delete /opt/magick-cache /
Be careful, after this command, your cached content is irrevocably lost.
MagickCache security is not crytographically strong. Instead it generates a unique hash for each resource ensuring the resource ID cannot be discovered. A resource is accessible to both the user of the cache and the cache owner. They are also accessible to anyone with sufficient privileges to access the MagickCache disk path.