To run Python code you need to ... 1. Choose the appropriate/best language version 2. Find/install an interpreter to meet that language version need ([[#`py install`]]) 3. Create a virtual environment that uses the selected interpreter ([[#`py venv`]]) 4. Install the code's dependencies ([[#`py ready`]]) 5. Run the code ([[#`py run`]]) # Goals - [ ] Easiest way to [[#`py launch` (or `py exec`)|launch an interpreter]] - [x] Select the newest version - [x] Specify the version - [ ] Support the major OSs - [x] Linux - [x] macOS - [ ] [[#Windows support]] - [ ] Registry - [ ] Microsoft Store - [x] From a virtual environment - [ ] Install an interpreter (requires [[Prebuilt binaries|Relocatable builds]]) - [ ] [[Virtual environment location]] - [ ] [Inline script metadata](https://packaging.python.org/en/latest/specifications/inline-script-metadata/) for `requires-python` only? - [ ] Easiest way to [[#`py list`|list interpreters]] - [ ] Fancy/human-readable - [ ] JSONL - [ ] Easiest way to [[#`py run`|run code]] 🏃 - [ ] [Inline script metadata](https://packaging.python.org/en/latest/specifications/inline-script-metadata/) - [ ] [[requires-python spec]] - [ ] `.py` files - [ ] Path - [ ] URL - [ ] Directories via `__main__.py`? - [ ] Zip files via `__main__.py`? - [ ] `pylock.toml` (takes care of interpreter selection and virtual environment creation, even if you do specify config details) - [ ] Scripts next to `pylock.toml`? - [ ] Directories w/ `__main__.py`? - [ ] Modules (`-m`) based on cwd? - [ ] `bin/` based on cwd? - [ ] Zip file? - [ ] `pyproject.toml` (takes care of interpreter selection and virtual environment creation, even if you do specify config details) - [ ] Write out lock file? - [ ] Scripts? - [ ] Directories w/ `__main__.py`? - [ ] Modules (`-m`) based on cwd? - [ ] Zip file? - [ ] `bin/` based on cwd? - [ ] Install Python on demand (requires [[Prebuilt binaries|Relocatable builds]]) - [ ] `py x` shorthand (like from `chmod`, plus nod to pipx)? - [ ] `venv` + `ready` for more manual control while keeping installation of packages simple (requires way to launch interpreter only) - [ ] Easiest way to [[#`py tool`|run tools]] 🧰 - [ ] Install - [ ] Run (`py x`?) # Ideas ## Automatic virtual environment management The goal is to make virtual environments an implementation detail to help prevent users from messing up their install and to simplify their workflows. There are two situations to consider: _ephemeral_ environments (like for inline script metadata), and _long-lasting_ environments (e.g. `.venv/` in a project). Probably need to make sure that if they manage to install something into an ephemeral virtual environment it's considered different than when it was first created. Simplest thing would be to create a single-use lock file for the environment and verify that everything matches. The next possibility that's even more robust is to keep projects unpacked somewhere, have them be read-only and recreate the virtual environments from scratch every time via filesystem links. Determining which ephemeral virtual environment to use could be done by hashing the inputs used to do the install. Ephemeral virtual environments should be occasionally cleaned up. Maybe every 30 days or past the same day of the month the following month? It could be done in a background thread for all ephemeral virtual environments. There should also be a way to forcibly recreate the virtual environment to make sure to use the newest project versions. ## Subcommands [Proposal for user-contributed subcommands](https://github.com/brettcannon/python-launcher/discussions/222) - Make each command a feature of the crate to control what commands are available? - Complex commands should decompose to other commands ### `py exec` #### Other tools - [`pymanager exec`](https://peps.python.org/pep-0773/#exec-subcommand) #### Purpose Same as bare `py` in terms of semantics/purpose (or skip and push people towards [[#`py run`]]?). Workflow tools do not have an equivalent command. Everyone else has their `run` command execute the command as found in the virtual environment if a path is not provided, i.e. `run python`. #### Notes - `--version`/`--python`: specify the Python version, much like e.g. `-3.12` - `uv` uses `--python` - `--ignore-venv`/`--exclude-venv`/`--skip-venv`/`--omit-venv`/`--no-venv`: exclude virtual environments from the list of potential interpreters ([poll on flag name](https://fosstodon.org/@brettcannon/114394209663606574)) - `--`: All flags following are passed on to the interpreter - Used by [`uv run`](https://docs.astral.sh/uv/reference/cli/#uv-run) - Not necessary if flags required upfront before hitting a path or unrecognized flag - `-m`: Equivalent to `-m` to the interpreter - Pragmatic shortcut to minimize need to specify `--` for the `-m` common case - Argument parsing stops once a path, `-m`, or `--` is found ### `py list` #### Other tools - [`hatch python show`](https://hatch.pypa.io/latest/cli/reference/#hatch-python-show) - [`pdm python list`](https://pdm-project.org/en/latest/reference/cli/#list_2) - [`uv python list`](https://docs.astral.sh/uv/reference/cli/#uv-python-list) - [`pipx interpreter list`](https://pipx.pypa.io/latest/docs/#pipx-interpreter) - [`pymanager list`](https://peps.python.org/pep-0773/#list-subcommand) #### Purpose List all of the discovered interpreters. ### Notes - [Discourse discussion](https://discuss.python.org/t/what-information-is-useful-to-know-statically-about-an-interpreter/25563) - [PEP 711 cross-post](https://discuss.python.org/t/pep-711-pybi-a-standard-format-for-distributing-python-binaries/25547/14?u=brettcannon) - `--format`: Specify the output format; all output in UTF-8 - `jsonl`: Print [interpreter details](https://github.com/brettcannon/python-launcher/discussions/168) using [JSON Lines](https://jsonlines.org/) - This should work with [`json -o` for Nu](https://www.nushell.sh/commands/docs/from_json.html) - Results are streamed as results are found instead of being sorted - Fields - `version` - `path` - `virtual`? - Can be inferred based on how the environment was found - `via` - `$VIRTUAL_ENV` - `.venv` - `$PATH` - `pretty` (default): Print interpreter details in a format designed to be human-readable - Include same details as the JSONL output - Sorted by how "good" the interpreter is - `--ignore-venv`: Ignore virtual environments (or default to `--include-venv`)? - Might be unnecessary or only useful via `--one` - `--one` - Only print the path to the "best" interpreter - https://github.com/python/cpython/issues/102522 would give more accurate version info for virtual environments ### `py install` #### Other tools - [pymanager install](https://peps.python.org/pep-0773/#install-subcommand) - [`uv python`](https://docs.astral.sh/uv/reference/cli/#uv-python) - [`poetry python`](https://python-poetry.org/docs/cli/#python) - [`hatch python`](https://hatch.pypa.io/latest/cli/reference/#hatch-python) - [`pdm python`](https://pdm-project.org/latest/reference/cli/#python) ([`pdm py`](https://pdm-project.org/latest/reference/cli/#py)) #### Purpose Install/manage Python interpreters. #### Notes - Worry about anything other than CPython? - PyPy? - Any builds other than the native CPU? - WASI? ### `py venv` #### Other tools - [`pdm venv`](https://pdm-project.org/latest/reference/cli/#venv) #### Purpose Create a virtual environment. #### Notes Requires [[Virtual environment location]] landing. - Create a virtual environment in `.venv` next to `pyproject.toml` (this should be the same as what `py run` would do if it was pointed at a project to create a virtual environment) - Provide a way to point to a preexisting or already active virtual environment - Be able to specify an alternative location - Provide the command to create the shell code to activate the virtual environment? - Probably not because if I did things right it won't be necessary - `--with-pip`; default - Install pip either into the virtual environment or use the `.pyz` - `=cached`: write a `.pth` that connects to `pip.pyz` as maintained for [[#`py ready`]] and then add a `bin/pip` command - `=local`: install into the virtual environment - Can you just unpack `pip.pyz` and then add `bin/pip`? - Is this necessary? Should users be told to rely on `-m venv` if they need a separate copy of pip? - Only necessary if you need a specific version of pip or running a Python version too old for the current `pip.pyz` - Both cases are rather unique - Need an easy way to run entry points? - Other tools use their `run` command to accomplish this - Could have [[#`py tool`]] , [[#`py exec`]], or [[#`py run`]] handle this ### `py ready` #### Other tools - [`pdm sync`](https://pdm-project.org/latest/reference/cli/#sync) - [`uv sync`](https://docs.astral.sh/uv/reference/cli/#uv-sync) - [`poetry sync`](https://python-poetry.org/docs/cli/#sync) - [Hatch syncs the environments automatically](https://hatch.pypa.io/latest/environment/#dependencies) whenever you run a command ### Purpose Get everything ready to run code. #### Notes - Create a virtual environment as necessary - Automatically install dependencies - Specify extras, dependency groups, and/or lock file (see [[#`py run`]]) - Does **not** install pip and setuptools - Keep a cached copy of the latest `pip.pyz` around instead - `--editable` to do an editable install - `--watch` to watch the file used for installing dependencies to automatically recreate the environment - Some option to opt into installing sdists and source trees (i.e. [`pip install --prefer-binary`](https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-only-binary) by default) - Specify indexes - Have a specific flag to opt out of PyPI ### `py run` #### Other tools - [`pdm run`](https://pdm-project.org/en/latest/reference/cli/#run) - [`poetry run`](https://python-poetry.org/docs/cli/#run) - [`hatch run`](https://hatch.pypa.io/latest/cli/reference/#hatch-run) - [`uv run`](https://docs.astral.sh/uv/reference/cli/#uv-run) - [`pipx run`](https://pipx.pypa.io/latest/docs/#pipx-run) #### Purpose For simplest way to execute code in an implicit virtual environment (i.e. the Python interpreter is an implementation detail). #### Notes Need to consider [[Caching app files while being ignored by backup apps]] for the implicit virtual environments. - `--group`/`--with`: Specifies the dependency group to use (only applicable when a directory path is specified, maybe zip file if it ships with a `pyproject.toml`) - Care about extras, e.g. `--extra` or implicitly specified via `--group`/`--with`? - `uv` uses `--extra` - Making it the same name as for dependency groups helps transition from an extra to a group when a project makes such a move - If a file path is specified: - If the file is a zip file, do anything special like look for inline script metadata in `__main__.py`? A lock file? - If [inline script metadata](https://packaging.python.org/en/latest/specifications/inline-script-metadata/) is specified, use that - Otherwise assume metadata of no dependencies and any Python version - If a directory path is specified: - For convenience when a [directory contains a `__main__.py` file](https://docs.python.org/3/using/cmdline.html) - Use inline script metadata, `pyproject.toml`, or `pylock.toml`? - Provide a way to specify whether `launch` or `run` are the default when no subcommand specified? - Don't read the shebang; only rely on the metadata to limit the Python version - Execute what's in `bin/` for the virtual environment if command name is specified? - Fall back to `-m` if nothing found in `bin/` matching the name (and it wasn't a valid path)? ### `py tool` #### Other tools - [`pipx run`](https://pipx.pypa.io/latest/docs/#pipx-run) - [`uv tool`](https://docs.astral.sh/uv/reference/cli/#uv-tool) #### Purpose Install and/or run tools from e.g. PyPI. This would provide functionality similar to `npm install -g`. #### Notes Trickiest bit is the UI for installing versus running such that running is very short when you don't want to bother installing (`py x`?). Another is whether some version restriction could be picked up from `pyproject.toml` or `pylock.toml`? - `--from`: specify what project to install - Support specifiers - `--spec` from pipx - [`--from` from uv](https://docs.astral.sh/uv/reference/cli/#uv-tool-run--from) - Fallback to `-m` if an entry point with the specified name isn't found? - How long to keep around ephemeral environments? - pipx is 14 days - [uv keeps using the cached version indefinitely](https://docs.astral.sh/uv/concepts/tools/#tool-environments) - Can use `@latest` to always check for a newer version - Provide a way to force a fresh install? - Use installed version from `install` or keep separate from `run`? - uv shares, but only when a specifier wasn't used to get a specific thing for `run` ### `py completions` #### Other tools - [`pdm completion`](https://pdm-project.org/latest/reference/cli/#completion) #### Purpose Print the completion script for the specified shell. #### Notes It would be a command instead of a static file so that it can be accurate for what interpreters are installed. But if the shells provide a way to do it dynamically then this might not be worth it. ## Library crates Wait until after any `py install` command happens and seeing if relocatable builds could be used to create a Python Launcher written in Python. - Create `locatepython` crate (or `findpython`), based around JSON schema - Add Windows support - Share across `py`, PVSC, and `uv` - Shebang reading? - Inline metadata parsing? - `requires-python` handling? - Cached pip handling? ## Windows support - [Virtual environments](https://github.com/brettcannon/python-launcher/issues/57) - [`PATH`](https://github.com/brettcannon/python-launcher/issues/14) - [Registry](https://github.com/brettcannon/python-launcher/issues/15) - [Microsoft Store](https://github.com/brettcannon/python-launcher/issues/16)