File mirror
Your containers most likely depend on some files you pull from the internet: dependencies hosted on GitHub, build toolchains, CLI tooling from your cloud vendor, and many more. For example, in your Dockerfile:
When you aim for reliable and stable builds this is problematic as files might change or get deleted at any time. There's zero guarantee that anything hosted on a URL will remain available; nor that it will always return the same file.
StableBuild solves this by operating a file mirror that acts as a pull-through cache to other servers. You prefix any URL that you download with your StableBuild file mirror URL, e.g.:
The first time this file is requested we'll fetch it from awscli.amazonaws.com, and then store the file in StableBuild. The next time you request this file we'll serve it from cache. The file is now immutable, and you'll always get the exact same file back - regardless whether the file was changed or deleted on the host server.
As different containers might require different versions of this file, you specify a cache key (above
my-first-tutorial). Every cache key is its own separate cache.
That's it. Anything you download from the internet is now stable and reliable. 🎉
Authentication / HTTP headers
If you need access to a URL that requires authentication - for example an API key - you can pass the key in through an HTTP header. This is forwarded to the original server whenever the URL is not in cache. For example:
Afterwards this URL is in cache, and the URL is the cache key. This means that any headers sent are not interpreted anymore. The URL above will return the same content, even if you change the API key.
Basic authentication
If you use basic authentication, you can pass in the username/password in the URL. If you have the following URL: https://httpbin.org/basic-auth/admin/secret (this is a real URL) - which has as credentials username admin and password secret you can encode the username/password in the URL like this:
Tips & tricks
Deleting files from cache
Cached file are listed under Dashboard > File mirror. Go to "Cached files", click on the cache key, and find your URL there. Click the three dots, then Delete to delete the file from cache. On subsequent requests the file will be re-fetched.
Caching GitHub repositories
If you need access to a GitHub repository you might have some code like:
StableBuild cannot cache git commands, but you can instead download a ZIP file with this commit from GitHub which can be cached. E.g.:
Uploading files manually
If you want to store local files in the file mirror:
Go to Dashboard > File mirror
Under "Cached files", either select a cache key or create a new cache key using the blue "New cache key" button.
Click "Upload file", select your file and a name.
Afterwards, find the uploaded file under "Cached files", select the three dots and select "Copy URL". You can then use that URL in your Dockerfile or build script.
Storage limits
Any cached file is subject to storage limits (or overage fees on paid plans). You can see your current storage usage on Dashboard. To determine if files are no longer fetched from cache (and thus whether you could delete them), you can look at the raw traffic logs under Dashboard > Billing > Raw usage logs.
Large URLs are processed asynchronously
When you make a HTTP request to StableBuild's file mirror, we usually cache the file while you download the first time. But in some cases we process files asynchronously. This happens:
If the file returned is >5GB.
If the file is >100MB, but no
Content-Lengthheader is set.
(This does not apply to files uploaded in the UI, they are always immediately processed.)
When this happens it might take a few minutes (or even more, depending on the size) before the URLs are cached in StableBuild. If this happens you'll see an hourglass icon next to the URL under "Cached files":
Note that because this file is processed asynchronously, the file will be fetched a second time (from a different machine in a different data center).
Last updated
