I have 2 SmartOS servers at home with a total of 17 zones
running persistently - a majority of these provisioned with the latest
LTS support release
of pkgsrc: 2014Q4. All zones are setup to download a similar set of "bootstrap"
packages to make them ready to be used. Packages like
git, etc. I pull to all
of my zones upon creation.
The problem, however, is my home internet is not the fastest - it usually can pull
packages at around 100KB/s.
gcc alone are responsible for over
100MB of compiled binary data, meaning these initial package downloads can take
over 15 minutes per new zone.
A possible solution to speed up initial package downloads I investigated was to
rsync the entire pkgsrc tree to a local server, and host it over an internal
HTTP server. This, however, required a massive amount of storage dedicated
to packages I am almost guaranteed to never use. I could have just pulled
2014Q4, but then I would have been required to add new releases whenever I
wanted to upgrade, and keep around the old packages until all my old zones have
Instead, I created a Node.JS HTTP caching server to proxy
HEAD requests to Joyent's pkgsrc server, and cache the data on the
local filesystem. The requests will be proxied if the file does not exist locally,
and will be dual-written to the requesting client, and the local filesystem. Subsequent
requests for the same resource will be streamed from the local filesystem without
every making an outbound request.
The program is written to be generic - it allows it to act as a caching proxy to any website, and also allows the user to specify a regex to test a URL when deciding if the request should be cached, or just proxied directly with no cache.
To spin up a server on SmartOS to front the Joyent pkgsrc servers install the
npm install -g fs-caching-server
Then create the directory that will be used locally to cache packages as they are lazily retrieved (any directory will work).
mkdir -p /data/pkgsrc-proxy
And finally, create an SMF manifest to start the service - all configuration parameters will be set as environmental variables.
<?xml version='1.0'?> <!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'> <service_bundle type='manifest' name='export'> <service name='application/pkgsrc-proxy' type='service' version='0'> <create_default_instance enabled='true'/> <dependency name='dep0' grouping='require_all' restart_on='error' type='service'> <service_fmri value='svc:/milestone/multi-user:default'/> </dependency> <exec_method name='start' type='method' exec='fs-caching-server &' timeout_seconds='10'> <method_context working_directory='/data/pkgsrc-proxy'> <method_credential user='nobody' group='nobody'/> <method_environment> <envvar name='PATH' value='/opt/local/bin:/opt/local/sbin:/usr/bin:/usr/sbin:/opt/custom/bin'/> <envvar name='FS_CACHE_PORT' value='8000'/> <envvar name='FS_CACHE_HOST' value='0.0.0.0'/> <envvar name='FS_CACHE_URL' value='http://pkgsrc.joyent.com'/> <envvar name='FS_CACHE_DEBUG' value='true'/> </method_environment> </method_context> </exec_method> <exec_method name='stop' type='method' exec=':kill' timeout_seconds='30'/> <template> <common_name> <loctext xml:lang='C'>Joyent pkgsrc proxy and caching service</loctext> </common_name> </template> </service> </service_bundle>
The service will cache data in its
CWD by default, which is set in the
If a regex is not specified as
FS_CACHE_REGEX it will default to
/\.(png|jpg|jpeg|css|html|js|tar|tgz|tar\.gz)$/ which is sufficient for
caching pkgsrc packages.
Finally, configure any SmartOS zones to use the new server. Modify
/opt/local/etc/pkgin/repositories.conf to point to the new server.
# http://pkgsrc.joyent.com/packages/SmartOS/2014Q4/x86_64/All http://10.0.1.50:8000/packages/SmartOS/2014Q4/x86_64/All
10.0.1.50:8000 with whatever host and port was used earlier)
And rebuild the database
rm -f /var/db/pkgin/pkgin.db pkgin -y up
And your done! Any new packages installed will be pulled through the caching proxy.