jupyterlite_core
#
Config#
an observable configuration object for the JupyterLite lifecycle
- class jupyterlite_core.config.LiteBuildConfig(**kwargs)#
the description of a JupyterLite build
This is most likely to be configured: - from environment variables - in a pyproject.toml - from the command line With direct instantiation a distant last place.
This is conceptually similar in scale to jupyter_server_config.json, and will piggy-back off of the {sys.prefix}/share/jupyter_{notebook,server}_config.d/ loader paths
-
app_archive:
Path
# JUPYTERLITE_APP_ARCHIVE
- Type:
The app archive to use. env
-
apps:
Tuple
[str
]# the Lite apps to explicitly include in build e.g. lab, tree, repl
-
base_url:
str
# JUPYTERLITE_BASE_URL
- Type:
[serve] the prefix to use. env
-
cache_dir:
Path
# A cache folder
-
contents:
Tuple
[Path
]# Contents to add and index
-
disable_addons:
Tuple
[str
]# should be a dict
- Type:
skip loading entry_point for these addons. TODO
-
extra_file_types:
dict
# extra JupyterLab-compatible file types for the server and browser
-
extra_http_headers:
dict
# the HTTP headers to add to default headers on all served responses
-
extra_ignore_contents:
Tuple
[str
]# Additional path regular expressions that should never be included as contents
-
federated_extensions:
Tuple
[str
]# Local paths or URLs in which to find federated_extensions
-
file_types:
dict
# JupyterLab-compatible file types for the server and browser
-
http_headers:
dict
# the HTTP headers to add to all served responses
-
ignore_contents:
Tuple
[str
]# Path regular expressions that should never be included as contents
-
ignore_sys_prefix:
Union
[bool
,Tuple
[str
]]# ignore components from sys.prefix
-
lite_dir:
Path
# JUPYTERLITE_DIR
- Type:
The root folder of a JupyterLite project. env
-
no_libarchive:
bool
# Don’t detect and use libarchive-c for higher performance and more archives
-
no_sourcemaps:
bool
# Strip all sourcemaps from applications and extensions
Remove any shared packages not used by –apps
-
output_archive:
Path
# JUPYTERLITE_OUTPUT_ARCHIVE
- Type:
Archive to create. env
-
output_dir:
Path
# JUPYTERLITE_OUTPUT_DIR
- Type:
Where to build the JupyterLite site. env
-
port:
int
# JUPYTERLITE_PORT
- Type:
[serve] the port to (insecurely) expose on http
- Type:
//127.0.0.1. env
-
settings_overrides:
Tuple
[str
]# Specific overrides.json to include
-
source_date_epoch:
Optional
[int
]# Trigger reproducible builds, clamping timestamps to this value
-
app_archive:
Constants#
Well-known (and otherwise) constants used by JupyterLite
- jupyterlite_core.constants.ADDON_ENTRYPOINT = 'jupyterlite.addon.v0'#
the extension point for addons, including core
- jupyterlite_core.constants.ALL_APP_ARCHIVES = [PosixPath('/home/docs/checkouts/readthedocs.org/user_builds/jupyterlite/checkouts/latest/py/jupyterlite-core/jupyterlite_core/jupyterlite-app-0.4.3.tgz')]#
all of the archives
- jupyterlite_core.constants.ALL_JSON = 'all.json'#
commonly-used filenames for response fixtures, e.g. settings
- jupyterlite_core.constants.API_CONTENTS = 'api/contents'#
the Jupyter API route for Contents API
- jupyterlite_core.constants.C_LOCALE = 'C'#
a locale for reproducible file sorting
- jupyterlite_core.constants.DEFAULT_FILE_TYPES = {'base64': {'gzip': [['.tgz', '.gz', '.gzip'], ['application/gzip']], 'ico': [['.ico'], ['image/x-icon']], 'jpeg': [['.jpeg', '.jpg'], ['image/jpeg']], 'pdf': [['.pdf'], ['application/pdf']], 'png': [['.png'], ['image/png']], 'wasm': [['.wasm'], ['application/wasm']], 'wheel': [['.whl'], ['octet/stream', 'application/x-wheel+zip']]}, 'json': {'geojson': [['.geojson'], ['application/geo+json']], 'ipynb': [['.ipynb'], ['application/x-ipynb+json']], 'jsmap': [['.map'], ['application/json']], 'json': [['.json'], ['application/json']]}, 'text': {'css': [['.css'], ['text/css']], 'csv': [['.csv'], ['text/csv']], 'fasta': [['.fasta'], ['text/plain']], 'html': [['.html'], ['text/html']], 'ical': [['.ical', '.ics', '.ifb', '.icalendar'], ['text/calendar']], 'js': [['.js', '.mjs'], ['application/javascript']], 'manifest': [['.manifest'], ['text/cache-manifest']], 'md': [['.md', '.markdown'], ['text/markdown']], 'plain': [['.txt'], ['text/plain']], 'py': [['.py'], ['text/x-python', 'application/x-python-code']], 'svg': [['.svg'], ['image/svg+xml']], 'toml': [['.toml'], ['application/toml']], 'vue': [['.vue'], ['text/plain']], 'xml': [['.xml'], ['application/xml']], 'yaml': [['.yaml', '.yml'], ['application/x-yaml']]}}#
enough file types to serve all our demo files
- jupyterlite_core.constants.DEFAULT_OUTPUT_DIR = '_output'#
Needs a better canonical location
- jupyterlite_core.constants.DISABLED_EXTENSIONS = 'disabledExtensions'#
configuration key for disabled extensions
- jupyterlite_core.constants.EXTENSION_TAR = ('.tgz', '.tar.bz2', '.tar.gz')#
known compressed tar extensions
- jupyterlite_core.constants.EXTENSION_ZIP = ('.whl', '.zip', '.conda')#
known zip extensions
- jupyterlite_core.constants.FEDERATED_EXTENSIONS = 'federated_extensions'#
configuration key for prebuilt extensions
- jupyterlite_core.constants.HOOKS = ['status', 'init', 'build', 'check', 'serve', 'archive']#
our doit task-based plugin system
- jupyterlite_core.constants.HOOK_PARENTS = {'archive': 'build', 'build': 'init', 'check': 'build', 'serve': 'build'}#
the name of the previous hook
- jupyterlite_core.constants.INDEX_HTML = 'index.html'#
a predictably-serveable HTML file
- jupyterlite_core.constants.JSON_FMT = {'indent': 2, 'sort_keys': True}#
default arguments for normalized JSON
- jupyterlite_core.constants.JUPYTERLITE_IPYNB = 'jupyter-lite.ipynb'#
our configuration file
- jupyterlite_core.constants.JUPYTERLITE_JSON = 'jupyter-lite.json'#
our configuration file
- jupyterlite_core.constants.JUPYTERLITE_SCHEMA = 'jupyterlite.schema.v0.json'#
our schema
- jupyterlite_core.constants.JUPYTER_CONFIG_DATA = 'jupyter-config-data'#
a script DOM ID on most jupyter pages
- jupyterlite_core.constants.LITE_PLUGIN_SETTINGS = 'litePluginSettings'#
the top-level key for lite plugin settings
- jupyterlite_core.constants.MOD_DIRECTORY = 493#
a default permission for directories
- jupyterlite_core.constants.MOD_FILE = 420#
a default permission for files
- jupyterlite_core.constants.NPM_SOURCE_DATE_EPOCH = 499162500#
this is arrived at by inspection
- jupyterlite_core.constants.OVERRIDES_JSON = 'overrides.json'#
settings overrides. used JupyterLab build system, usually goes in $PREFIX/share/jupyter/lab/
- jupyterlite_core.constants.PACKAGE_JSON = 'package.json'#
the canonical location of labextension metadata
- jupyterlite_core.constants.PHASES = ['pre_', '', 'post_']#
the lifecycle stages inside a hook
- jupyterlite_core.constants.REQUIREMENTS_TXT = 'requirements.txt'#
the generally-used listing of pip requirements
- jupyterlite_core.constants.SETTINGS_FILE_TYPES = 'fileTypes'#
configuration key for file types
- jupyterlite_core.constants.SETTINGS_OVERRIDES = 'settingsOverrides'#
configuration key for extension settings overrides
- jupyterlite_core.constants.SHA256SUMS = 'SHA256SUMS'#
output equivalent to sha256sum * for providing a local bill-of-data
- jupyterlite_core.constants.SHARE_LABEXTENSIONS = 'share/jupyter/labextensions'#
the canonical location within an env (or archive) for labextensions
- jupyterlite_core.constants.SOURCEMAPS = ['.js.map', '.mjs.map']#
extensions to be considered sourcemaps
- jupyterlite_core.constants.SOURCE_DATE_EPOCH = 'SOURCE_DATE_EPOCH'#
a canonical environment variable for triggering reproducible builds
- jupyterlite_core.constants.UTF8 = {'encoding': 'utf-8'}#
the encoding for pretty much every file written and read by jupyterlite
App#
the JupyterLite CLI App(s)
- class jupyterlite_core.app.BaseLiteApp(**kwargs)#
TODO: An undescribed app
- property aliases#
Get CLI aliases, including ones provided by addons.
- emit_alias_help()#
Yield the lines for alias part of the help.
copied from:
Unlike the upstream, this also takes Addon classes (and their parents) into consideration.
- property flags#
Get CLI flags, including ones provided by addons.
- class jupyterlite_core.app.DescribedMixin#
a self-describing mixin
- class jupyterlite_core.app.LiteApp(**kwargs)#
build ready-to-serve (or -publish) JupyterLite sites
- class jupyterlite_core.app.LiteArchiveApp(**kwargs)#
build a JupyterLite app archive, which can be used as a baseline
- class jupyterlite_core.app.LiteBuildApp(**kwargs)#
build a JupyterLite site, including user content
- class jupyterlite_core.app.LiteCheckApp(**kwargs)#
verify a JupyterLite site, using available schema and rules
- class jupyterlite_core.app.LiteDoitApp(**kwargs)#
Run the doit command
- start()#
Start the whole thing
- class jupyterlite_core.app.LiteInitApp(**kwargs)#
initialize a JupyterLite site from an app archive baseline
- class jupyterlite_core.app.LiteListApp(**kwargs)#
describe a JupyterLite site
- class jupyterlite_core.app.LiteRawDoitApp(**kwargs)#
use the full doit CLI, see https://pydoit.org/contents.html
tell jupyter to not parse the arguments with –, e.g.
jupyter-lite doit – –help
- parse_command_line(argv=None)#
Parse the command line arguments.
- class jupyterlite_core.app.LiteServeApp(**kwargs)#
serve a JupyterLite site, using best available HTTP server
- class jupyterlite_core.app.LiteStatusApp(**kwargs)#
report about what a JupyterLite build _might_ do
- class jupyterlite_core.app.LiteTaskApp(**kwargs)#
run a doit task, optionally with –force
- property flags#
CLI flags, including some custom ones.
- force#
forget previous runs of task and re-run from the beginning
- start()#
Start the whole thing
- class jupyterlite_core.app.ManagedApp(**kwargs)#
An app with a LiteManager that can do some config fixing
- start()#
Start the whole thing
- jupyterlite_core.app.lite_flags = {'debug': ({'Application': {'log_level': 10}}, 'set log level to logging.DEBUG (maximize logging output)'), 'ignore-sys-prefix': ({'LiteBuildConfig': {'ignore_sys_prefix': True}}, 'Do not copy anything from sys.prefix'), 'no-libarchive': ({'LiteBuildConfig': {'no_libarchive': True}}, 'Do not try to use libarchive-c for archive operations'), 'no-sourcemaps': ({'LiteBuildConfig': {'no_sourcemaps': True}}, 'Strip all sourcemaps from applications and extensions'), 'no-unused-shared-packages': ({'LiteBuildConfig': {'no_unused_shared_packages': True}}, 'Remove shared packages not used by --apps'), 'y': ({'JupyterApp': {'answer_yes': True}}, 'Answer yes to any questions instead of prompting.')}#
the total set of flags from discovered addons
Manager#
Manager for JupyterLite
- class jupyterlite_core.manager.LiteManager(**kwargs)#
a manager for building jupyterlite sites
This primarily handles the business of mapping _addons_ to
doit
_tasks_, and then calling thedoit
API.- create_addons()#
initialize addons from entry_points
if populated,
disable_addons
will be consulted.Subclasses may choose to override this, but are encouraged to include the default addons
- doit_run(task, *args, raw=False)#
run a subset of the doit command line
- initialize()#
perform one-time inialization of the manager
- parsed_extra_args#
extra CLI args unused by the
LiteManager
- strict#
if True, stop the current workflow on the first error
- task_prefix#
a prefix appended to all tasks generated by this manager
Addons#
Archive#
a JupyterLite addon for generating app archives which can be used as input
- class jupyterlite_core.addons.archive.ArchiveAddon(**kwargs: Any)#
Adds contents from the
lite_dir
to theoutput_dir
, creates API outputIf
--source-date-epoch
(SDE) is set, a number of features will be enabled to improve reproducibility of the final artifact. In addition to timestamps newer than SDE being “clamped” to SDE, this will also adjust some permissions inside the tarball- archive(manager)#
add all files created prior to
pre_archive
to an archive
- filter_tarinfo(tarinfo)#
apply best-effort entropy fixes to give more reproducible archives
- log_archive(tarball, prefix='')#
print some information about an archive
- make_archive_stdlib(tarball, root, members)#
actually build the archive.
- this takes longer than any other hook
while this pure-python implementation needs to be maintained, a
libarchive
-based build might be preferable for e.g. CI performance.
an npm-compatible
.tgz
is the only supported archive format, as this is compatible with the upstreamwebpack
build and its native packaged format.
- setlocale(name)#
Context manager for changing the current locale
Base#
- class jupyterlite_core.addons.base.BaseAddon(**kwargs: Any)#
A base class for addons to the JupyterLite build chain
Subclassing this is optional, but provides some useful utilities
- copy_one(src, dest)#
copy one Path (a file or folder)
- dedupe_federated_extensions(config)#
update a federated_extension list in-place, ensuring unique names.
- delete_one(*src)#
delete… somethings
- extract_one(archive, dest)#
extract the contents of an archive to a path.
- fetch_one(url, dest)#
fetch one file
TODO: enable other backends, auth, etc.
- get_lite_config_paths()#
Yield all config paths that exist in the
lite_dir
.- Return type:
Generator
[Path
,None
,None
]
- get_lite_plugin_settings(config_path, plugin_id)#
Get the plugin settings from a config path.
The keys follow the JupyterLab settings naming convention, of module and identifier e.g. :rtype:
Dict
[str
,Any
]@jupyterlite/contents:plugin
- get_output_config_paths()#
Yield all config paths that _might_ exist in the
output_dir
.- Return type:
Generator
[Path
,None
,None
]
- property log#
Logger or LoggerAdapter instance
- merge_jupyter_config_data(config, in_config)#
merge well-known
jupyter-config-data
fields
- merge_one_jupyterlite(out_path, in_paths)#
write the
out_path
with the merge content ofin_paths
, where all are validjupyter-lite.*
files.
- set_lite_plugin_settings(config_path, plugin_id, settings)#
Overwrite the plugin settings for a single plugin in a config path.
- Return type:
None
- property should_use_libarchive_c#
should libarchive-c be used (if available)?
- task(**task)#
Ensure a
doit
task is well-formed.Task names may not include the
=
character - arbitrarily replace with--
- timestamp_one(path)#
adjust the timestamp to be –source-date-epoch for files newer than then
Contents#
a JupyterLite addon for Jupyter Server-compatible contents
- class jupyterlite_core.addons.contents.ContentsAddon(**kwargs: Any)#
Adds contents from the
lite_dir
to theoutput_dir
, creates API output- build(manager)#
perform the main user build of pre-populating
/files/
- check(manager)#
verify that all Contents API is valid (sorta)
- property file_src_dest#
the pairs of contents that will be copied
these are processed in reverse order, such that only the last path wins
- maybe_add_one_path(path, root=None)#
add a file or folder’s contents (if not ignored)
- one_contents_path(output_file_dir, api_path)#
A lazy reuse of a
jupyter_server
Contents API generator
- patch_listing_timestamps(listing, sde=None)#
clamp a contents listing’s times to
SOURCE_DATE_EPOCH
- post_build(manager)#
create a Contents API index for each subdirectory in
/files/
- status(manager)#
yield some status information about the state of contents
- class jupyterlite_core.addons.contents.DateTimeEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)#
A custom date-aware JSON encoder
- default(o)#
Implement this method in a subclass such that it returns a serializable object for
o
, or calls the base implementation (to raise aTypeError
).For example, to support arbitrary iterators, you could implement default like this:
def default(self, o): try: iterable = iter(o) except TypeError: pass else: return list(iterable) # Let the base class default method raise the TypeError return super().default(o)
- jupyterlite_core.addons.contents.isoformat(dt)#
a small helper to user
Z
for UTC ISO strings
Federated Extensions#
a JupyterLite addon for supporting federated_extensions
- class jupyterlite_core.addons.federated_extensions.FederatedExtensionAddon(**kwargs: Any)#
sync the as-installed federated_extensions and update jupyter-lite.json
- property archive_cache#
where archives will go in the cache
- build(manager)#
yield a doit task to copy each local extension into the output_dir
- copy_all_federated_extensions(unarchived)#
actually copy all federated extensions found in a folder.
- copy_conda2_extensions(conda_pkg)#
copy the labextensions from a local, nested
.conda
package
- copy_one_env_extension(pkg_json)#
yield tasks to copy one unpacked on-disk extension from sys.prefix into the output dir
- copy_one_extension(pkg_json)#
yield a task to copy one unpacked on-disk extension from anywhere into the output dir
- copy_one_federated_extension(pkg_json)#
actually copy one labextension from an extracted archive
- copy_one_folder_extension(path)#
yield a task to copy one extension from the given path
- copy_simple_archive_extensions(archive)#
yield tasks to extract and copy the labextensions from a local simple archive
- ensure_federated_settings(manager, lab_extensions, all_federated_json)#
ensure settings from federated extensions are aggregated in a single file
- env_extensions(root)#
a list of all federated extensions
- property ext_cache#
where extensions will go in the cache
- extra_labextensions_path#
Extra paths to look for federated JupyterLab extensions
- get_federated_settings(extension)#
get the settings for a federated extension
- is_prebuilt(pkg_json)#
verify this is an actual pre-built extension, containing load information
- property output_extensions#
where labextensions will go in the output folder
- patch_jupyterlite_json(jupyterlite_json)#
add the federated_extensions to jupyter-lite.json
- post_build(manager)#
update the root jupyter-lite.json, and copy each output theme to each app
- post_init(manager)#
handle downloading of federated extensions
- pre_build(manager)#
yield a doit task to copy each federated extension into the output_dir
- resolve_one_extension(path_or_url, init)#
yield tasks try to resolve one URL or local folder/archive as a (set of) federated_extension(s)
Lite#
a JupyterLite addon for jupyterlite-specific tasks
- class jupyterlite_core.addons.lite.LiteAddon(**kwargs: Any)#
ensure jupyterlite files have been merged, and validate them
- build(manager)#
merge jupyter-lite.json into the output_dir
- check(manager)#
apply schema validation to all jupyter-lite.json in the output_dir
- property lite_files#
all the source jupyter-lite.* files
Report#
a JupyterLite addon for generating hashes
- class jupyterlite_core.addons.report.ReportAddon(**kwargs: Any)#
update static listings of the site contents in various formats
having these in various formats down the line can be handy for various publishing tasks
- pre_archive(manager)#
generate a hash file of all files in the distribution.
As this is relatively expensive for hundreds of files, this is performed as late as possible, while still providing some useful publishing / QA features.
- TODO: develop some contract with the frontend in relation to this file,
or a derivative, as it has precisely the right information for certain cache tasks.
- property sha256sums#
The location of the hashfile.
Serve#
a JupyterLite addon for serving
- class jupyterlite_core.addons.serve.ServeAddon(**kwargs: Any)#
Settings#
a JupyterLite addon for supporting extension settings
- class jupyterlite_core.addons.settings.SettingsAddon(**kwargs: Any)#
sync the as-installed overrides.json and update jupyter-lite.json
- property output_extensions#
where labextensions will go in the output folder
- patch_one_overrides(jupyterlite_json, overrides_json)#
update and normalize settingsOverrides
- post_build(manager)#
add settings from overrides.json per app where they exist
Static#
a JupyterLite addon for jupyterlab core
- class jupyterlite_core.addons.static.StaticAddon(**kwargs: Any)#
Copy the core “gold master” artifacts into the output folder
- app_archive#
The path to a custom npm-style tarball (e.g. with package/package.json). This may alternately be specified with the $JUPYTERLITE_APP_ARCHIVE environment variable.
- init(manager)#
unpack and copy the tarball files into the output_dir
- post_init(manager)#
maybe remove sourcemaps, or all static assets if an app is not installed
- pre_init(manager)#
well before anything else, we need to ensure that the output_dir exists and is empty (if the baseline tarball has changed)
manually remove unused webpack chunks from shared packages