CLI

import os, pathlib, tempfile, shutil, atexit, hashlib, pprint
from IPython.display import *
from IPython import get_ipython # needed for `jupyter_execute` because magics?
RTD = os.environ.get("READTHEDOCS")

The jupyter lite (or jupyter-lite) CLI provides tools for lifecycle of combining…

  • the core JupyterLite static assets

  • extra application features like Lab Extensions and settings

  • user-authored content like Notebooks

… into a ready-to-deploy (and optionally reproducible) Jupyter sites which require an HTTP server, but no application server.

Installation

!pip install jupyterlite
!jupyter lite --version
Requirement already satisfied: jupyterlite in /home/docs/checkouts/readthedocs.org/user_builds/jupyterlite/conda/latest/lib/python3.9/site-packages (0.1.0a14)
Requirement already satisfied: entrypoints in /home/docs/checkouts/readthedocs.org/user_builds/jupyterlite/conda/latest/lib/python3.9/site-packages (from jupyterlite) (0.3)
Requirement already satisfied: doit in /home/docs/checkouts/readthedocs.org/user_builds/jupyterlite/conda/latest/lib/python3.9/site-packages (from jupyterlite) (0.33.1)
Requirement already satisfied: jupyter_core>=4.7 in /home/docs/checkouts/readthedocs.org/user_builds/jupyterlite/conda/latest/lib/python3.9/site-packages (from jupyterlite) (4.8.1)
Requirement already satisfied: traitlets in /home/docs/checkouts/readthedocs.org/user_builds/jupyterlite/conda/latest/lib/python3.9/site-packages (from jupyter_core>=4.7->jupyterlite) (5.1.0)
Requirement already satisfied: pyinotify in /home/docs/checkouts/readthedocs.org/user_builds/jupyterlite/conda/latest/lib/python3.9/site-packages (from doit->jupyterlite) (0.9.6)
Requirement already satisfied: cloudpickle in /home/docs/checkouts/readthedocs.org/user_builds/jupyterlite/conda/latest/lib/python3.9/site-packages (from doit->jupyterlite) (2.0.0)
0.1.0a14

Extras

Some extra features of different addons have additional dependencies.

pip install jupyterlite[contents]  # jupyter_server for better contents API
pip install jupyterlite[serve]  # tornado for better local preview
pip install jupyterlite[lab]  # a known-compatible jupyterlab (entails `contents` and `serve`)
# TODO: [archive]  # use libarchive

Quick Start

Once installed, one can get to a locally-hosted, empty JupyterLite site with:

jupyter lite serve

This will:

  • init the baseline assets

  • build all extra assets

    • if a files/ folder is found, those will be copied into _output/files

      • … and indexed if jupyter_server is installed

  • serve the site locally and print out a URL

The Lite Dir

When you run jupyter lite commands, it assumes your current working directory is the partial contents of a JupyterLite site. You can override this with --lite-dir. By default, the built site will be created in _output, but can be overridden with --output-dir.

if "TMP_DIR" not in globals():
    TMP_DIR = pathlib.Path(tempfile.mkdtemp(prefix="_my_lite_dir_"))
    def clean():
        shutil.rmtree(TMP_DIR)
    atexit.register(clean)
os.chdir(TMP_DIR)
print(pathlib.Path.cwd())
/tmp/_my_lite_dir_5m63x94c

Well-known Files

Some files in your --lite-dir that have special meaning:

look in path…

… for file named

… and if found

.
./lab
./retro

jupyter-lite.{json,ipynb}

merge with static in _output/{path}/jupyter-lite.{json,ipynb}

.
./lab
./retro

overrides.json

merge with static in _output/{*}/jupyter-lite.json

./files/

*

copy verbatim to _output/files/* and index in /api/contents

See more about merging contents.

Usage

Common Parameters

parameter

description

default

environment variable

--lite-dir

configuration and content for the site

current working directory

JUPYTERLITE_DIR

--output-dir

where the hostable site will be created

_output

JUPYTERLITE_OUTPUT_DIR

--cache-dir

a cache directory for downloads

<lite_dir>/.cache

JUPYTERLITE_CACHE_DIR

--app-archive

an alternate site to base off of

bundled

--contents

directory to copy to _output/files/ and available as Contents

./files

--ignore-contents

patterns that should never be included in /files/ (even if found in lite-dir)

various

--output-archive

the path to the archive

<directory>-jupyterlite.tgz

JUPYTERLAB_OUTPUT_ARCHIVE

--port

port on 127.0.0.1 to serve the test server

8000

JUPYTERLITE_PORT

--base-url

the URL prefix to include before the site

/

JUPYTERLITE_BASE_URL

--source-date-epoch

optionally enable additional reproducible build measures (best-effort!)

SOURCE_DATE_EPOCH

--federated-extensions

paths to folders, pip/conda packages with extensions see note

--ignore-sys-prefix

don’t copy any contents, such as install labextensions, from sys.prefix

False

--settings-overrides

additional settings overrides to enable

All parameters may be configured via a jupyter_lite_config.json in the directory where jupyter lite is launched, or given via --config.

Hint

For an advanced example, see the configuration used for this documentation.

Help

The CLI provides its own documentation, under --help (or -h).

!jupyter lite --help
build ready-to-serve (or -publish) JupyterLite sites

Subcommands
===========
Subcommands are launched as `jupyter cmd [args]`. For information on using
subcommand 'cmd', do: `jupyter cmd -h`.

list
    describe a JupyterLite site
status
    report about what a JupyterLite build _might_ do
init
    initialize a JupyterLite site from an app archive baseline
build
    build a JupyterLite site, including user content
check
    verify a JupyterLite site, using available schema and rules
serve
    serve a JupyterLite site, using best available HTTP server
archive
    build a JupyterLite app archive, which can be used as a baseline
doit
    use the full doit CLI, see https://pydoit.org/contents.html

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to logging.DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=10]
--show-config
    Show the application's configuration (human-readable format)
    Equivalent to: [--Application.show_config=True]
--show-config-json
    Show the application's configuration (json format)
    Equivalent to: [--Application.show_config_json=True]
--generate-config
    generate default config file
    Equivalent to: [--JupyterApp.generate_config=True]
-y
    Answer yes to any questions instead of prompting.
    Equivalent to: [--JupyterApp.answer_yes=True]
--ignore-sys-prefix
    Do not copy any extensions from sys.prefix
    Equivalent to: [--LiteBuildConfig.ignore_sys_prefix=True]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--config=<Unicode>
    Full path of a config file.
    Default: ''
    Equivalent to: [--JupyterApp.config_file]
--app-archive=<CPath>
    The app archive to use. env: JUPYTERLITE_APP_ARCHIVE
    Default: traitlets.Undefined
    Equivalent to: [--LiteBuildConfig.app_archive]
--apps=<typedtuple-item-1>...
    the Lite apps: currently ['lab', 'retro']. Required: ['lab']
    Default: ()
    Equivalent to: [--LiteBuildConfig.apps]
--contents=<typedtuple-item-1>...
    Contents to add and index
    Default: ()
    Equivalent to: [--LiteBuildConfig.contents]
--ignore-contents=<tuple-item-1>...
    Path regular expressions that should never be included as contents
    Default: ()
    Equivalent to: [--LiteBuildConfig.ignore_contents]
--lite-dir=<CPath>
    The root folder of a JupyterLite project. env: JUPYTERLITE_DIR
    Default: traitlets.Undefined
    Equivalent to: [--LiteBuildConfig.lite_dir]
--output-dir=<CPath>
    Where to build the JupyterLite site. env: JUPYTERLITE_OUTPUT_DIR
    Default: traitlets.Undefined
    Equivalent to: [--LiteBuildConfig.output_dir]
--output-archive=<CPath>
    Archive to create. env: JUPYTERLITE_OUTPUT_ARCHIVE
    Default: traitlets.Undefined
    Equivalent to: [--LiteBuildConfig.output_archive]
--settings-overrides=<typedtuple-item-1>...
    Specific overrides.json to include
    Default: ()
    Equivalent to: [--LiteBuildConfig.settings_overrides]
--source-date-epoch=<CInt>
    Trigger reproducible builds, clamping timestamps to this value
    Default: 0
    Equivalent to: [--LiteBuildConfig.source_date_epoch]
--port=<CInt>
    [serve] the port to (insecurely) expose on http://127.0.0.1. env:
    JUPYTERLITE_PORT
    Default: 0
    Equivalent to: [--LiteBuildConfig.port]
--base-url=<Unicode>
    [serve] the prefix to use. env: JUPYTERLITE_BASE_URL
    Default: ''
    Equivalent to: [--LiteBuildConfig.base_url]

To see all available configurables, use `--help-all`.

Status

Always safe to run, this command provides an overview of what JupyterLite has been doing.

!jupyter lite status
static:jupyter-lite.json
.  pre_status:static:jupyter-lite.json
    tarball:  jupyterlite-app-0.1.0-alpha.14.tgz 16MB
    output:   /tmp/_my_lite_dir_5m63x94c/_output
    lite dir: /tmp/_my_lite_dir_5m63x94c
    apps:     ('lab', 'retro')
archive:archive
contents:contents
lite:jupyter-lite.json
serve:contents
settings:overrides
translation:translation
.  status:archive:archive
[LiteStatusApp] No archive (yet): _my_lite_dir_5m63x94c-jupyterlite.tgz
.  status:contents:contents
    contents: 0 files
.  status:lite:jupyter-lite.json
[LiteStatusApp]     jupyter-lite.(json|ipynb): 0 files
.  status:serve:contents
    will serve 8000 with: tornado
.  status:settings:overrides
    overrides.json: 0
.  status:translation:translation
    translation files: 0 files

List

Always safe to run, this command provides an overview of what JupyterLite might do.

TODO: improve on default output

!jupyter lite list
static:jupyter-lite.json
archive:archive
contents:contents
lite:jupyter-lite.json
serve:contents
settings:overrides
translation:translation
static:output_dir
static:unpack
federated_extensions:copy:ext:nbdime-jupyterlab
federated_extensions:copy:ext:@jupyter-widgets/jupyterlab-manager
translation:copy
federated_extensions:patch
serve:tornado
report:SHA256SUMS
archive:archive:_my_lite_dir_5m63x94c-jupyterlite.tgz
R archive                                                                       
R archive:archive:archive:_my_lite_dir_5m63x94c-jupyterlite.tgz                 generate a new app archive
R build                                                                         
R build:translation:copy                                                        create the translation data
R check                                                                         
R init                                                                          
R init:static:unpack                                                            unpack a 'gold master' JupyterLite from jupyterlite-app-0.1.0-alpha.14.tgz
R post_archive                                                                  
R post_build                                                                    
E post_build:federated_extensions:patch                                         ensure jupyter-lite.json includes the federated_extensions
R post_check                                                                    
R post_init                                                                     
R post_serve                                                                    
R post_status                                                                   
R pre_archive                                                                   
R pre_archive:report:SHA256SUMS                                                 hash all of the files
R pre_build                                                                     
R pre_build:federated_extensions:copy:ext:@jupyter-widgets/jupyterlab-manager   
R pre_build:federated_extensions:copy:ext:nbdime-jupyterlab                     
R pre_check                                                                     
R pre_init                                                                      
R pre_init:static:output_dir                                                    clean out the lite directory
R pre_serve                                                                     
R pre_status                                                                    
R pre_status:static:jupyter-lite.json                                           
R serve                                                                         
R serve:serve:tornado                                                           run server at http://127.0.0.1:8000/ for /tmp/_my_lite_dir_5m63x94c/_output
R status                                                                        
R status:archive:archive                                                        
R status:contents:contents                                                      
R status:lite:jupyter-lite.json                                                 
R status:serve:contents                                                         
R status:settings:overrides                                                     
R status:translation:translation                                                

Init

Copy all the static data to the --output-dir.

!jupyter lite init
static:jupyter-lite.json
.  pre_status:static:jupyter-lite.json
    tarball:  jupyterlite-app-0.1.0-alpha.14.tgz 16MB
    output:   /tmp/_my_lite_dir_5m63x94c/_output
    lite dir: /tmp/_my_lite_dir_5m63x94c
    apps:     ('lab', 'retro')
archive:archive
contents:contents
lite:jupyter-lite.json
serve:contents
settings:overrides
translation:translation
.  status:archive:archive
[LiteInitApp] No archive (yet): _my_lite_dir_5m63x94c-jupyterlite.tgz
.  status:contents:contents
    contents: 0 files
.  status:lite:jupyter-lite.json
[LiteInitApp]     jupyter-lite.(json|ipynb): 0 files
.  status:serve:contents
    will serve 8000 with: tornado
.  status:settings:overrides
    overrides.json: 0
.  status:translation:translation
    translation files: 0 files
static:output_dir
.  pre_init:static:output_dir
static:unpack
.  init:static:unpack

Build

Copy all the user-authored content to the --output-dir, and applies appropriate changes to e.g. generated Contents API responses.

Special well-known files will be merged appropriately, but generally, files that exist in the user directory will overwrite any existing content.

!jupyter lite build
static:jupyter-lite.json
.  pre_status:static:jupyter-lite.json
    tarball:  jupyterlite-app-0.1.0-alpha.14.tgz 16MB
    output:   /tmp/_my_lite_dir_5m63x94c/_output
    lite dir: /tmp/_my_lite_dir_5m63x94c
    apps:     ('lab', 'retro')
archive:archive
contents:contents
lite:jupyter-lite.json
serve:contents
settings:overrides
translation:translation
.  status:archive:archive
[LiteBuildApp] No archive (yet): _my_lite_dir_5m63x94c-jupyterlite.tgz
.  status:contents:contents
    contents: 0 files
.  status:lite:jupyter-lite.json
[LiteBuildApp]     jupyter-lite.(json|ipynb): 0 files
.  status:serve:contents
    will serve 8000 with: tornado
.  status:settings:overrides
    overrides.json: 0
.  status:translation:translation
    translation files: 0 files
static:output_dir
-- pre_init:static:output_dir
static:unpack
-- init:static:unpack
federated_extensions:copy:ext:nbdime-jupyterlab
federated_extensions:copy:ext:@jupyter-widgets/jupyterlab-manager
.  pre_build:federated_extensions:copy:ext:nbdime-jupyterlab
.  pre_build:federated_extensions:copy:ext:@jupyter-widgets/jupyterlab-manager
translation:copy
.  build:translation:copy
federated_extensions:patch
.  post_build:federated_extensions:patch

Serve

Serve the --output-dir on http://127.0.0.1:{--port=8000}{--base-url=/}.

Warning

This is not a production server. Please consider any of the deployment options before trying to make this something it isn’t.

!jupyter lite serve --help
serve a JupyterLite site, using best available HTTP server

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to logging.DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=10]
--show-config
    Show the application's configuration (human-readable format)
    Equivalent to: [--Application.show_config=True]
--show-config-json
    Show the application's configuration (json format)
    Equivalent to: [--Application.show_config_json=True]
--generate-config
    generate default config file
    Equivalent to: [--JupyterApp.generate_config=True]
-y
    Answer yes to any questions instead of prompting.
    Equivalent to: [--JupyterApp.answer_yes=True]
--force
    forget previous runs of task and re-run from the beginning
    Equivalent to: [--LiteTaskApp.force=True]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--config=<Unicode>
    Full path of a config file.
    Default: ''
    Equivalent to: [--JupyterApp.config_file]
--app-archive=<CPath>
    The app archive to use. env: JUPYTERLITE_APP_ARCHIVE
    Default: traitlets.Undefined
    Equivalent to: [--LiteBuildConfig.app_archive]
--apps=<typedtuple-item-1>...
    the Lite apps: currently ['lab', 'retro']. Required: ['lab']
    Default: ()
    Equivalent to: [--LiteBuildConfig.apps]
--contents=<typedtuple-item-1>...
    Contents to add and index
    Default: ()
    Equivalent to: [--LiteBuildConfig.contents]
--ignore-contents=<tuple-item-1>...
    Path regular expressions that should never be included as contents
    Default: ()
    Equivalent to: [--LiteBuildConfig.ignore_contents]
--lite-dir=<CPath>
    The root folder of a JupyterLite project. env: JUPYTERLITE_DIR
    Default: traitlets.Undefined
    Equivalent to: [--LiteBuildConfig.lite_dir]
--output-dir=<CPath>
    Where to build the JupyterLite site. env: JUPYTERLITE_OUTPUT_DIR
    Default: traitlets.Undefined
    Equivalent to: [--LiteBuildConfig.output_dir]
--output-archive=<CPath>
    Archive to create. env: JUPYTERLITE_OUTPUT_ARCHIVE
    Default: traitlets.Undefined
    Equivalent to: [--LiteBuildConfig.output_archive]
--settings-overrides=<typedtuple-item-1>...
    Specific overrides.json to include
    Default: ()
    Equivalent to: [--LiteBuildConfig.settings_overrides]
--source-date-epoch=<CInt>
    Trigger reproducible builds, clamping timestamps to this value
    Default: 0
    Equivalent to: [--LiteBuildConfig.source_date_epoch]
--port=<CInt>
    [serve] the port to (insecurely) expose on http://127.0.0.1. env:
    JUPYTERLITE_PORT
    Default: 0
    Equivalent to: [--LiteBuildConfig.port]
--base-url=<Unicode>
    [serve] the prefix to use. env: JUPYTERLITE_BASE_URL
    Default: ''
    Equivalent to: [--LiteBuildConfig.base_url]

To see all available configurables, use `--help-all`.

Check

Use all available mechanisms to verify that the build folder conforms to schema, etc.

!jupyter lite check
static:jupyter-lite.json
.  pre_status:static:jupyter-lite.json
    tarball:  jupyterlite-app-0.1.0-alpha.14.tgz 16MB
    output:   /tmp/_my_lite_dir_5m63x94c/_output
    lite dir: /tmp/_my_lite_dir_5m63x94c
    apps:     ('lab', 'retro')
archive:archive
contents:contents
lite:jupyter-lite.json
serve:contents
settings:overrides
translation:translation
.  status:archive:archive
[LiteCheckApp] No archive (yet): _my_lite_dir_5m63x94c-jupyterlite.tgz
.  status:contents:contents
    contents: 0 files
.  status:lite:jupyter-lite.json
[LiteCheckApp]     jupyter-lite.(json|ipynb): 0 files
.  status:serve:contents
    will serve 8000 with: tornado
.  status:settings:overrides
    overrides.json: 0
.  status:translation:translation
    translation files: 4 files
static:output_dir
-- pre_init:static:output_dir
static:unpack
-- init:static:unpack
federated_extensions:copy:ext:nbdime-jupyterlab
federated_extensions:copy:ext:@jupyter-widgets/jupyterlab-manager
-- pre_build:federated_extensions:copy:ext:nbdime-jupyterlab
-- pre_build:federated_extensions:copy:ext:@jupyter-widgets/jupyterlab-manager
translation:copy
.  build:translation:copy
federated_extensions:patch
-- post_build:federated_extensions:patch
lite:validate:jupyter-lite.json
lite:validate:lab/jupyter-lite.json
lite:validate:retro/jupyter-lite.json
lite:validate:retro/tree/jupyter-lite.json
lite:validate:retro/consoles/jupyter-lite.json
lite:validate:retro/edit/jupyter-lite.json
lite:validate:retro/notebooks/jupyter-lite.json
translation:validate:translation:all.json
.  check:lite:validate:jupyter-lite.json
.  check:lite:validate:lab/jupyter-lite.json
.  check:lite:validate:retro/jupyter-lite.json
.  check:lite:validate:retro/tree/jupyter-lite.json
.  check:lite:validate:retro/consoles/jupyter-lite.json
.  check:lite:validate:retro/edit/jupyter-lite.json
.  check:lite:validate:retro/notebooks/jupyter-lite.json
.  check:translation:validate:translation:all.json

Archive

Turn the output directory into a .tgz file. This is usually easier to move around than (sometimes) hundreds of files, and can be used as the baseline for future sites.

This command is relatively expensive, and is skipped for documentation purposes

!jupyter lite archive --help
if not RTD:
    !jupyter lite archive
build a JupyterLite app archive, which can be used as a baseline

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to logging.DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=10]
--show-config
    Show the application's configuration (human-readable format)
    Equivalent to: [--Application.show_config=True]
--show-config-json
    Show the application's configuration (json format)
    Equivalent to: [--Application.show_config_json=True]
--generate-config
    generate default config file
    Equivalent to: [--JupyterApp.generate_config=True]
-y
    Answer yes to any questions instead of prompting.
    Equivalent to: [--JupyterApp.answer_yes=True]
--force
    forget previous runs of task and re-run from the beginning
    Equivalent to: [--LiteTaskApp.force=True]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--config=<Unicode>
    Full path of a config file.
    Default: ''
    Equivalent to: [--JupyterApp.config_file]
--app-archive=<CPath>
    The app archive to use. env: JUPYTERLITE_APP_ARCHIVE
    Default: traitlets.Undefined
    Equivalent to: [--LiteBuildConfig.app_archive]
--apps=<typedtuple-item-1>...
    the Lite apps: currently ['lab', 'retro']. Required: ['lab']
    Default: ()
    Equivalent to: [--LiteBuildConfig.apps]
--contents=<typedtuple-item-1>...
    Contents to add and index
    Default: ()
    Equivalent to: [--LiteBuildConfig.contents]
--ignore-contents=<tuple-item-1>...
    Path regular expressions that should never be included as contents
    Default: ()
    Equivalent to: [--LiteBuildConfig.ignore_contents]
--lite-dir=<CPath>
    The root folder of a JupyterLite project. env: JUPYTERLITE_DIR
    Default: traitlets.Undefined
    Equivalent to: [--LiteBuildConfig.lite_dir]
--output-dir=<CPath>
    Where to build the JupyterLite site. env: JUPYTERLITE_OUTPUT_DIR
    Default: traitlets.Undefined
    Equivalent to: [--LiteBuildConfig.output_dir]
--output-archive=<CPath>
    Archive to create. env: JUPYTERLITE_OUTPUT_ARCHIVE
    Default: traitlets.Undefined
    Equivalent to: [--LiteBuildConfig.output_archive]
--settings-overrides=<typedtuple-item-1>...
    Specific overrides.json to include
    Default: ()
    Equivalent to: [--LiteBuildConfig.settings_overrides]
--source-date-epoch=<CInt>
    Trigger reproducible builds, clamping timestamps to this value
    Default: 0
    Equivalent to: [--LiteBuildConfig.source_date_epoch]
--port=<CInt>
    [serve] the port to (insecurely) expose on http://127.0.0.1. env:
    JUPYTERLITE_PORT
    Default: 0
    Equivalent to: [--LiteBuildConfig.port]
--base-url=<Unicode>
    [serve] the prefix to use. env: JUPYTERLITE_BASE_URL
    Default: ''
    Equivalent to: [--LiteBuildConfig.base_url]

To see all available configurables, use `--help-all`.

But let’s talk about a more reproducible asset.

# we clean out the TMP_DIR for the reproducibility examples
shutil.rmtree(TMP_DIR / "_output", ignore_errors=True)

Reproducible Archives

🛠️ This feature is a work-in-progress, and should not be relied upon by any production workflows Just Yet.

If --source-date-epoch is given, a number of measures will be taken to try to ensure that the output of jupyter lite archive, an npm-compatible tgz package, always returns a bit-for-bit reproducible build.

The most obvious change is that the modified time of each file “clamped” to that time. Some other changes:

  • file ownership will be reset

  • predictable sorting will be used

  • additional checks will be applied

Note

This is a shortcut for setting the environment variable SOURCE_DATE_EPOCH:

platform

command

Linux
MacOS

export SOURCE_DATE_EPOCH=<a timestamp>

Windows

set SOURCE_DATE_EPOCH=<a timestamp>

Python

os.environ.update(SOURCE_DATE_EPOCH, <a timestamp>)

if not "source_date_epoch" in globals():
    from datetime import datetime
    source_date_epoch = int(datetime.utcnow().timestamp())

print("SOURCE_DATE_EPOCH is", source_date_epoch)
SOURCE_DATE_EPOCH is 1634373858
if not RTD:
    !jupyter lite archive --source-date-epoch {source_date_epoch} --output-archive ./a.tgz

If we clear out our _output

if not RTD:
    shutil.rmtree(TMP_DIR / "_output", ignore_errors=True)
    pprint.pprint([*TMP_DIR.rglob("*")])

…and rebuild, we should always get the same file.

if not RTD:
    !jupyter lite archive --source-date-epoch {source_date_epoch} --output-archive ./b.tgz
if not RTD:
    a, b = [
        hashlib.sha256((TMP_DIR / f"{x}.tgz").read_bytes()).hexdigest() 
        for x in "ab"
    ]
    print("We built app archives with the SHA256SUMS of:\n", a, "\n", b)
    try:
        assert a == b, "We did not reproducibly build today.\n- {}\n- {}\n\n".format(a, b)
    except AssertionError as err:
        if shutil.which("diffoscope"):
            print("We did NOT reproducibly build today, checking in with `diffoscope`...")
            !diffoscope a.tgz b.tgz
        print("...but at least we tried REALLY hard!\n")

Miscellaneous

Merging Contents

In addition to the default files/ path, provide additional files to show in the File Manager or Tree when JupyterLite launches with --contents, which can be provided multiple times.

Note

  • If --contents is provided, the default value of --contents files will be ignored

  • If given multiple times, and child folder/file names would collide, the last path wins

Hint

For complex file layouts, a jupyter_lite_config.json is highly recommended.

Contents Example

Given a directory like:

my-lite-dir/
  files/
    README.md
    a.md
  more-files/
    README.md
    b.md

Running jupyter lite build without any arguments will yield:

my-lite-dir/
  _output/
    README.md
    b.md

Whereas jupyter lite build --contents more-files --contents files will yield:

my-lite-dir/
  _output/
    files/
      README.md # this will be from my-lite-dir/files
      a.md
      b.md

And jupyter lite build --contents files --contents more-files will yield:

my-lite-dir/
  _output/
    files/
      README.md # this will be from my-lite-dir/more-files
      a.md
      b.md

conda packages

While --federated-extensions support the .tar.bz2 created by most conda packages, there are some issues:

  • anaconda.org uses non-standard HTTP headers to S3 buckets to provide packages

  • the conda-forge channel provides all of its builds as GitHub releases, and can be predictably transformed, e.g.

https://anaconda.org/conda-forge/jupyterlab_widgets/1.0.0/download/noarch/jupyterlab_widgets-1.0.0-pyhd8ed1ab_1.tar.bz2
                                                                   |      |
                                                                   |      +---------------------------------------------+
                                                                   v      v                                             v
         https://github.com/conda-forge/releases/releases/download/noarch/jupyterlab_widgets-1.0.0-pyhd8ed1ab_1.tar.bz2/jupyterlab_widgets-1.0.0-pyhd8ed1ab_1.tar.bz2